IP-Ranges Updater

Back in March I wrote about a command line script filter-ip-ranges that can parse the Amazon-published ip-ranges.json file and look up address ranges by region, service or IP address. That’s handy for the occasional manual use or for creating or updating some whitelists or Security Groups through aws-cli.

But running an aws-cli script from cron on some EC2 instance is so old school!

Much better way is to have this functionality in a Lambda function and let it update your Security Groups or Route Tables automatically.

Welcome IP-Ranges Updater project!

Obviously I didn’t develop this Lambda just for the sake of it. Obviously I had a problem that needed solving. Actually two problems.

I wanted to restrict access to a web server only through CloudFront. In my case to enforce WAF (Web Application Firewall), which at the time wasn’t available for ELB/ALB (now it is). One of the ways to achieve that is to add all the CloudFront CIDRs to the to the EC2 or ELB Security Group.

• Solution: Let IP-Ranges Updater populate the Security Group rules and keep it up to date when new CloudFront CIDRs are added.

One of my customers has very strict policies regarding outbound access from VPCs. EC2 instance don’t have direct outside access and everything has to go through a Proxy with URL whitelist. Unfortunately there are services like Fargate or Cognito that don’t work without direct outside access. For these special cases we got a security exemption to create dedicated subnets in the VPC with NAT but that NAT could only be used to access the Amazon IPs in our region, nothing else!

• Solution: Use IP-Ranges Updater to add our local region’s IP Ranges to the new subnets’ Route Tables with target being the NAT gateway.

Head over to the IP-Ranges Updater on GitHub and check out the README file for detailed installation instructions.

Here I’ll add only a couple of notes…

• Just like with the command-line tool filter-ip-ranges you can filter prefixes by region or service. In this case the filter is in JSON format as described in the README.
• The Lambda Python file can be called from the shell to facilitate development of the JSON filter.

  ~/aws-ipranges-updater $ ./ipranges_updater/lambda.py \
                           --json '[{"region":"ap-southeast-2","services":["S3"]}]'
  SELECTED: 4 prefixes
  52.92.52.0/22 ap-southeast-2 AMAZON S3
  52.95.128.0/21 ap-southeast-2 AMAZON S3
  54.231.248.0/22 ap-southeast-2 AMAZON S3
  54.231.252.0/24 ap-southeast-2 AMAZON S3
  

• In the config.sh file comment out settings that you don’t need. For example  if you are only going to update Ingress rules in Security Groups but not Route Tables or Security Group Egress rules comment out ROUTE_TABLES, RT_TARGET and SG_EGRESS_PORTS lines.

The Lambda is scheduled to run once a day, at some random point in the day. That’s fine because ip-ranges.json doesn’t change that often and even after AWS adds new CIDRs to the file it takes some time before they start using them.

It is however possible to run it as soon as and only when ip-ranges.json is updated. Every time Amazon updates the file they publish a SNS notification to a SNS Topic in us-east-1 – we can subscribe to it and trigger our run from there. Check out the use-sns branch in GitHub for an implementation.

Unfortunately on a few occasions I found that the ip-ranges.json file was not yet readable when the SNS Topic delivered the notification which essentially meant that I missed out on those updates. Running it periodically once a day seems to be more reliable.

• Either way you may want to run the newly created Lambda manually for the first time after deployment to populate your Security Groups or Route Tables without having to wait a day for the scheduled run or for the file update if you are using SNS.

That’s all 🙂