Stop heading to Docker Hub Limits: Use Amazon ECR Public Gallery
A while ago, Docker Hub implemented some limits on the Pull requests into their registry.
To this day, this never bothered me that much to the point that I needed to do something about it.
Well, the day that I needed to do something about it arrived. I’m setting up a Kong Gateway with Datadog images as sidecars. Both images are pulled from the Docker Hub. However, since the Kong Gateway container failed to start, when I was able to fix the configuration of the ECS Task Definition (because it keeps trying to start the containers until it does), I had reached the Docker Hub limits for the pulling of the Datadog image.
When you get into a situation similar to this one, you will see this error on the AWS Console:
As devs, we get easily get used to doing the things the same way until we can’t do it anymore. And this was my relationship with Docker Hub so far. Because this error is blocking me from finishing my Jira Task, I needed to figure out some workaround for this issue.
For this situation, I could think of three ways to fix it:
Creating your own Public/Private ECR Registry and push the image manually
Use ECR Pull Through Cache with a Docker Hub API Key.
Use AWS ECR Public Registry
The easier one is to use the ECR Public Registry that was created just for this problem that Docker Hub created. Back in 2020, AWS posted this blog post where it stated that they would be creating a Public Registry without the Pull Limits that Docker Hub would implement, and advised the customers to change to use it in a coming future.
And today was the day that I went to the registry page for the first time:
Based on the docs, the Amazon ECR Public Gallery does not have a hard limit on how many pulls you can do it1, it just limits the requests per second that you can do:
AND if you want to keep a cache of the image that you are pulling from AWS Registry, you can configure Pull Through Cache easily.
So I just got the Datadog Agent registry address:
And added to my Terraform. Now my container starts. And everything is right again.
When following those instructions, you are now free of depending on Docker Hub for your Cloud stack.
That’s all folks!
Do you have time for more? Read my blog post about:
https://docs.aws.amazon.com/AmazonECR/latest/public/public-service-quotas.html