Updating AWS CloudFront IP to Nginx

20/05/2020

CloudFront become a very common CDN/Reverse proxy nowadays because of their high availability and easy to use. So if you place a CloudFront as a proxy server in front of your Nginx web server, than the Nginx web server not able to get the real customer IP address. Same thing will happen if you using cloudFlare as your CDN (will discuss this in other post)

Nginx using the http_realip_module to read the real client IP. To make this thing to work, there are few step that we need to be done.

Install the jq command in your server
Create the following shell script in your server
Add the execute permission for your shell script
Create the cronjob to run the script daily or hourly, depend on you
Include the cloudFront IP list to your nginx.conf file
Reload the nginx service
Test it

Installing jq command

$ sudo apt install jq

Because the IP range file which we download from AWS site is json format, so we need the jq command to act as a query language for us to filter the data that we needed.

Create the shell script to pull CloudFront IP

$ sudo vim updateCloudFrontIpRange.sh

#!/bin/bash
CLOUDFRONT_IP_RANGES_FILE_PATH="/etc/nginx/cloudfront.conf"
WWW_GROUP="www-data"
WWW_USER="www-data"

CLOUDFRONT_REMOTE_FILE="https://ip-ranges.amazonaws.com/ip-ranges.json"
CLOUDFRONT_LOCAL_FILE="/var/tmp/cloudfront-ips.json"

if [ -f /usr/bin/fetch ];
then
    fetch $CLOUDFRONT_REMOTE_FILE --no-verify-hostname --no-verify-peer -o $CLOUDFRONT_LOCAL_FILE --quiet
else
    wget -q $CLOUDFRONT_REMOTE_FILE -O $CLOUDFRONT_LOCAL_FILE --no-check-certificate
fi

echo "# Amazon CloudFront IP Ranges" > $CLOUDFRONT_IP_RANGES_FILE_PATH
echo "# Generated at $(date) by $0" >> $CLOUDFRONT_IP_RANGES_FILE_PATH
echo "" >> $CLOUDFRONT_IP_RANGES_FILE_PATH

cat $CLOUDFRONT_LOCAL_FILE  | jq -r '.prefixes[] | select(.service=="CLOUDFRONT") | .ip_prefix' | while read i; do echo "set_real_ip_from ${i};"  >> $CLOUDFRONT_IP_RANGES_FILE_PATH; done;

echo "real_ip_header X-Forwarded-For;" >> $CLOUDFRONT_IP_RANGES_FILE_PATH
echo "real_ip_recursive on;" >> $CLOUDFRONT_IP_RANGES_FILE_PATH
echo "" >> $CLOUDFRONT_IP_RANGES_FILE_PATH

chown $WWW_USER:$WWW_GROUP $CLOUDFRONT_IP_RANGES_FILE_PATH

rm -rf $CLOUDFRONT_LOCAL_FILE

service nginx reload

The above script is actually fetch the CloudFront IP range json file from AWS site and using the jq command to query the json file and select all the IP which belong to CloudFront, loop over the list of IP and insert into the file at /etc/nginx/cloudfront.conf (you may need to change it base on your Nginx installation).

We need to defines trusted IP addresses that are known to send correct replacement addresses. Typically we add upstream servers IP address. The syntax is:

set_real_ip_from ipv4_addresss;
set_real_ip_from ipv6_address;
set_real_ip_from sub/net;
set_real_ip_from CIDR;

# Amazon CloudFront IP Ranges                                                                                                                                                                                                                    
# Generated at Wed May 20 12:44:22 +08 2020 by ./cloudfront-ip-ranges-updater.sh                                                                                                                                                                 
set_real_ip_from 120.52.22.96/27;
set_real_ip_from 205.251.249.0/24;
set_real_ip_from 180.163.57.128/26;
set_real_ip_from 204.246.168.0/22;
set_real_ip_from 205.251.252.0/23;
set_real_ip_from 54.192.0.0/16;
set_real_ip_from 204.246.173.0/24;
set_real_ip_from 120.253.240.192/26;
set_real_ip_from 116.129.226.128/26;
set_real_ip_from 99.86.0.0/16;
set_real_ip_from 205.251.200.0/21;
set_real_ip_from 223.71.71.128/25;
set_real_ip_from 13.32.0.0/15;
set_real_ip_from 120.253.245.128/26;
set_real_ip_from 13.224.0.0/14;
set_real_ip_from 70.132.0.0/18;
set_real_ip_from 210.51.40.0/24;
set_real_ip_from 13.249.0.0/16;
set_real_ip_from 205.251.208.0/20;
set_real_ip_from 58.254.138.0/25;
set_real_ip_from 116.129.226.0/25;
set_real_ip_from 52.222.128.0/17;
set_real_ip_from 64.252.128.0/18;
set_real_ip_from 205.251.254.0/24;
set_real_ip_from 71.152.0.0/17;
set_real_ip_from 216.137.32.0/19;
set_real_ip_from 204.246.172.0/24;
set_real_ip_from 120.52.39.128/27;
set_real_ip_from 118.193.97.64/26;
set_real_ip_from 223.71.71.96/27;
set_real_ip_from 130.176.0.0/16;
set_real_ip_from 54.240.128.0/18;
set_real_ip_from 205.251.250.0/23;
set_real_ip_from 180.163.57.0/25;
set_real_ip_from 52.46.0.0/18;
set_real_ip_from 223.71.11.0/27;
set_real_ip_from 52.82.128.0/19;
set_real_ip_from 54.239.128.0/18;
set_real_ip_from 36.103.232.128/26;
set_real_ip_from 52.84.0.0/15;
set_real_ip_from 111.51.66.0/24;
set_real_ip_from 143.204.0.0/16;
set_real_ip_from 144.220.0.0/16;
set_real_ip_from 120.52.153.192/26;
set_real_ip_from 119.147.182.0/25;
set_real_ip_from 120.232.236.0/25;
set_real_ip_from 54.182.0.0/16;
set_real_ip_from 58.254.138.128/26;
set_real_ip_from 120.253.245.192/27;
set_real_ip_from 54.239.192.0/19;
set_real_ip_from 120.52.12.64/26;
set_real_ip_from 99.84.0.0/16;
set_real_ip_from 54.230.0.0/16;
set_real_ip_from 52.124.128.0/17;
set_real_ip_from 204.246.164.0/22;
set_real_ip_from 13.35.0.0/16;
set_real_ip_from 204.246.174.0/23;
set_real_ip_from 36.103.232.0/25;
set_real_ip_from 119.147.182.128/26;
set_real_ip_from 118.193.97.128/25;
set_real_ip_from 120.232.236.128/26;
set_real_ip_from 204.246.176.0/20;
set_real_ip_from 120.253.241.160/27;
set_real_ip_from 13.124.199.0/24;
set_real_ip_from 35.167.191.128/26;
set_real_ip_from 18.200.212.0/23;
set_real_ip_from 99.79.169.0/24;
set_real_ip_from 52.15.127.128/26;
set_real_ip_from 34.223.12.224/27;
set_real_ip_from 54.233.255.128/26;
set_real_ip_from 13.54.63.128/26;
set_real_ip_from 13.59.250.0/26;
set_real_ip_from 3.234.232.224/27;
set_real_ip_from 52.66.194.128/26;
set_real_ip_from 13.228.69.0/24;
set_real_ip_from 64.252.64.0/18;
set_real_ip_from 18.216.170.128/25;
set_real_ip_from 3.231.2.0/25;
set_real_ip_from 52.220.191.0/26;
set_real_ip_from 34.232.163.208/29;
set_real_ip_from 35.162.63.192/26;
set_real_ip_from 34.223.80.192/26;
set_real_ip_from 34.226.14.0/24;
set_real_ip_from 13.113.203.0/24;
set_real_ip_from 34.195.252.0/24;
set_real_ip_from 52.52.191.128/26;
set_real_ip_from 52.56.127.0/25;
set_real_ip_from 34.216.51.0/25;
set_real_ip_from 52.199.127.192/26;
set_real_ip_from 52.212.248.0/26;
set_real_ip_from 13.210.67.128/26;
set_real_ip_from 35.158.136.0/24;
set_real_ip_from 52.57.254.0/24;
set_real_ip_from 52.78.247.128/26;
set_real_ip_from 52.47.139.0/24;

real_ip_header X-Forwarded-For;
real_ip_recursive on;

This is what being added to your /etc/nginx/cloudfront.conf

Add the execute permission for your file

$ sudo chmod +x updateCloudFrontIpRange.sh
$ sudo ./updateCloudFrontIpRange.sh

For the first time you create this script, you need to execute the script at least one manually to download the IP list to your server.

Create cronjob to run your script

$ crontab -e

0 0 * * 0 ~/updateCloudFrontIpRange.sh

Insert to above cronjob schedule to your crontab editor, the schedule will be run at every Sunday 00:00. If you wish to change schedule time, free feel to change base on your use case, https://crontab.guru is a very good site where you can test your cron schedule expressions website.

Include the IP list into your Nginx conf

$ sudo vim /etc/nginx/nginx.conf

http {
...
...
include /etc/nginx/cloudfront.conf;
include /etc/nginx/conf.d/*.conf;
...
}

$ sudo service nginx reload

Edit the nginx.conf using your favorite text editor, my preference is vim. Insert the “include /etc/nginx/cloudfront.conf;“ just above “include /etc/nginx/conf.d/*.conf;”.

After all done, just reload the nginx service to refresh the configuration that we just apply.

Test the configuration

$ tail -f /var/log/nginx/access.log

You may try to access your site which host behind CloudFront now and check the nginx access log to see is that you get the real client IP or not.

If everything is correct, than you should see your real client IP instead of CloudFront IP

139.1.2.3 - - [07/May/2020:12:16:24 +0000] "GET / HTTP/1.1" 200 3032 "https://www.google.com/" "Amazon CloudFront"
142.2.3.4 - - [07/May/2020:12:16:25 +0000] "GET /style.css HTTP/1.1" 200 399 "Amazon CloudFront"
1.2.3.4 - - [07/May/2020:12:16:25 +0000] "GET /images/logo.png HTTP/1.1" 200 34001 "Amazon CloudFront"

The same concept also apply for other CDN provider such as CloudFlare, Stackpath and other, for those CDN, you should be able to get their list of IP range from their website and manually insert into the Nginx configuration file.

For those people who hosted the Nginx behind the AWS Application Load Balance (ALB) you may also need to include the load balance IP into the list, or else they Nginx will capture the ALB IP instead of CDN or real client IP.

To know more about the Nginx http real IP module, you may refer to their documentation at here.

Updating AWS CloudFront IP to Nginx

Installing jq command

Create the shell script to pull CloudFront IP

Add the execute permission for your file

Create cronjob to run your script

Include the IP list into your Nginx conf

Test the configuration

Like this:

Related

Leave a ReplyCancel reply

Recent Posts

Statistics

Updating AWS CloudFront IP to Nginx

Installing jq command

Create the shell script to pull CloudFront IP

Add the execute permission for your file

Create cronjob to run your script

Include the IP list into your Nginx conf

Test the configuration

Share this:

Like this:

Related

Related articles

Google Drive for desktop to be replaced with Backup and Sync app by March next year

Country blocking using Nginx with IP2Location Module

Leave a ReplyCancel reply

Hosting private static content in S3 using AWS Cloudfront

Passing client User-Agent from CloudFront to origin