Auto update CloudFront IP into security group using Lambda

AWS CloudFront (CDN) provide a better performance and low latency to the website visitor by caching the content at the EDGE location which closer to the visitor location. CloudFront not only provide the performance improvement to the end visitor, it’s also provide the lower cost of content delivery from your origin to the end visitor devices.

Because of the nature of the EDGE location caching of the content, the CloudFront will pull the content from the origin directly from the EDGE location which we are not able hardcoded the IP address where the server will come to pull content.

If your origin is application load balance (ALB) or EC2, than you may need to whitelist the range of IP in your security group in order to make sure the requester is a CloudFront server IP but not other visitor or attacker. AWS publishes these IP ranges in JSON format so that you can create networking configurations that use them. These ranges are separated by service and region, which means you’ll only need to allow IP ranges that correspond to CloudFront.

In the pass, you may need to copy and paste the IP list into the security group in order to whitelist the IP, but what if the IP range change after that? You may need to manually update the list again, another way is you write a cron job to keep checking on the list on any changing, than update your security group, end up you may need to manage another host or schedule in order to make your list up to date.

Both the method is not optimise as actually the AWS is publishing the SNS topic on the updated IP changed.

In order to utilise the SNS for the IP update, there are few step need to accomplish the task:

• Create 2 security group for http (optional additional 2 security group for https)
• Create the tags for the security group
• Create the policy and role for lambda
• Create the lambda function to update the list
• Add the trigger for the lambda by listen to SNS topic

Create security group

Go to EC2 console and click on the security group

Create 2 security group for HTTP, (optional another 2 security group for HTTPS), add the following 3 tags for each of the security group

Global ingress security group (tags)

• Name: cloudfront_g
• AutoUpdate: true
• Protocol: http (or any other protocol which you wish to whitelist from CloudFront)

Regional ingress security group (tags)

• Name: cloudfront_r
• AutoUpdate: true
• Protocol: http (or any other protocol which you wish to whitelist from CloudFront)

If you wish the whitelist both the http and https, than you may need create 4 security group. Another issue to be aware is that by default, security group have the limit of 60 inbound and 60 outbound rules allow per security group, so in order to make the auto update to work successfully, you may need to increase the limit by send the email to AWS team.

Create an IAM policy and execution role for the Lambda function

The following policy provide the permission for lambda to update the security group.

To create the policy

1. Go to IAM console
2. Click on Policy
3. Click on Create Policy
4. Select JSON
5. Copy the following JSON policy and paste it in the text area
6. Click on review policy
7. Give a meaningful name for your policy and we will use this policy to create a new role for Lambda

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupIngress"
      ],
      "Resource": "*"
    }
  ]
}

After the policy done created, now we need to create a new role for Lambda to assumed. Now we can create a new role from the policy

1. Go to IAM console
2. Click on role
3. Click on create role
4. Select type of trusted entity –> AWS Service
5. Choose used case –> Lambda
6. Click on next at the bottom
7. Filter the policy name that you just created above
8. Click on add tags and review the roles

Create your lambda function

Once the roles done created, now it’s time to create your lambda functions. To create the lambda function, the step as per following:

1. Go to lambda console
2. Click on create function
3. Select Author from scratch as we will create the function by copy and paste the code from AWS git repo
4. Enter the meaningful function name for your lambda
5. Select Runtime as Python 3.6
6. On the execution role part, click on use an existing roles and filter the role name that we just created early
7. Create on create functions to proceed next
8. Insert the python code in the code editor, you may download the script from the AWS Sample Code git repo
9. Save the functions

Test the lambda function

Until here, we already successfully created the lambda function, now it’s time to test and initialise the security group.

1. In the Lambda console on the Functions page, choose your function, choose the Actions drop-down menu, and then Configure test event.
2. Enter the following as your sample event, which will represent an SNS notification.

{
  "Records": [
    {
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:EXAMPLE",
      "EventSource": "aws:sns",
      "Sns": {
        "SignatureVersion": "1",
        "Timestamp": "1970-01-01T00:00:00.000Z",
        "Signature": "EXAMPLE",
        "SigningCertUrl": "EXAMPLE",
        "MessageId": "95df01b4-ee98-5cb9-9903-4c221d41eb5e",
        "Message": "{\"create-time\": \"yyyy-mm-ddThh:mm:ss+00:00\", \"synctoken\": \"0123456789\", \"md5\": \"7fd59f5c7f5cf643036cbd4443ad3e4b\", \"url\": \"https://ip-ranges.amazonaws.com/ip-ranges.json\"}",
        "Type": "Notification",
        "UnsubscribeUrl": "EXAMPLE",
        "TopicArn": "arn:aws:sns:EXAMPLE",
        "Subject": "TestInvoke"
      }
    }
  ]
}

3. After you’ve added the test event, click Save and test. Your Lambda function will be invoked, and you should see log output at the bottom of the console similar to the following.

Updating from https://ip-ranges.amazonaws.com/ip-ranges.json
MD5 Mismatch: got 2e967e943cf98ae998efeec05d4f351c expected 7fd59f5c7f5cf643036cbd4443ad3e4b: Exception
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 29, in lambda_handler
    ip_ranges = json.loads(get_ip_groups_json(message['url'], message['md5']))
  File "/var/task/lambda_function.py", line 50, in get_ip_groups_json
    raise Exception('MD5 Missmatch: got ' + hash + ' expected ' + expected_hash)
Exception: MD5 Mismatch: got 2e967e943cf98ae998efeec05d4f351c expected 7fd59f5c7f5cf643036cbd4443ad3e4b

You will see a message indicating there was a hash mismatch. Normally, a real SNS notification from the IP Ranges SNS topic will include the right hash, but because our sample event is a test case representing the event, you will need to update the sample event manually to have the expected hash.

4. Just replace the 7fd59f5c7f5cf643036cbd4443ad3e4b to 2e967e943cf98ae998efeec05d4f351c in your test event script (it may be difference for your case), this is to bypass the MD5 checksum. In the actual environment, this error will not occur.
5. Click on save and test, your lambda function will be invoke and it will update your security group as well. You also need to make sure that your security group limit has been increase from the default 60 inbound rule per security group up to at least 200 rule per security group.

At this time, you should get your security group updated with the current CloudFront IP list.

Configure lambda functions’ trigger

Once the security group is updated and the lambda function is working fine, it’s time to add in the auto update features for your lambda functions to be trigger.

In your lambda function Designer section, click on add trigger, than insert the SNS topic from AWS and click on the save.

arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged

You should get your lambda functions to be trigger whenever there is any changing on the AWS IP range.

In short, this step is to create a listener lambda functions which listening to any update of the AWS IP range. By using lambda, there is no any infrastructure or additional resources to manage with, it’s actually can save up our cost and ensure our IP list is keeping update.