Troubleshooting “dial tcp: lookup docker on x.x.x.x:53: no such host” Error When Pulling docker:dind from AWS ECR in GitLab CI


2 views

When migrating from Docker Hub to AWS ECR for GitLab CI/CD pipelines, many developers encounter a specific DNS resolution failure with the docker-in-docker (dind) service. The error manifests when trying to authenticate with ECR from within the CI environment:

error during connect: Post http://docker:2375/v1.24/auth: dial tcp: lookup docker on 8.8.8.8:53: no such host

In a typical GitLab CI setup using docker:dind, the service container needs to:

  • Resolve the 'docker' hostname to connect to the Docker daemon
  • Authenticate with ECR before pulling images
  • Maintain proper network connectivity between containers

The standard docker:dind image from Docker Hub includes special networking configurations that automatically handle the 'docker' hostname resolution. However, when pulling the same image from ECR:

  • The internal DNS resolution for the service name 'docker' fails
  • The default /etc/hosts entries differ from the Docker Hub version
  • Network aliases aren't automatically configured

Here's a working configuration for your .gitlab-ci.yml:

build-push:
  stage: package
  image: public.ecr.aws/x/x
  services:
    - name: public.ecr.aws/x/docker-dind:20.10
      alias: docker
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""

For cases where the alias doesn't work, you can manually modify /etc/hosts:

before_script:
  - echo "172.17.0.1 docker" >> /etc/hosts
  - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_DOCKER_REGISTRY_URL

When troubleshooting, these commands are invaluable:

# Check container networking
docker network inspect bridge

# Verify DNS resolution
nslookup docker

# Test connectivity
nc -zv docker 2375

# View Docker daemon logs
cat /var/log/docker.log

When using this approach:

  • Ensure your ECR repository policies are properly configured
  • Use temporary AWS credentials in your CI environment
  • Consider adding network-level restrictions in your VPC

When migrating Docker images from Docker Hub to AWS ECR for GitLab CI/CD pipelines, a specific issue emerges with the Docker-in-Docker (DinD) service. The same Docker image that works perfectly when pulled from Docker Hub fails authentication when pulled from AWS public ECR.

build-push:
  stage: package
  image: public.ecr.aws/x/x
  services:
  - public.ecr.aws/x/docker-dind:20.10

The core error manifests when trying to authenticate with ECR:

$ aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_DOCKER_REGISTRY_URL
> Logging in to Docker registry...
> error during connect: Post http://docker:2375/v1.24/auth: dial tcp: lookup docker on 8.8.8.8:53: no such host

This indicates the Docker client inside the container cannot resolve the hostname "docker" which should point to the DinD service.

The /etc/hosts file reveals the actual service addressing:

$ cat /etc/hosts
127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2  public.ecr.aws__*
172.17.0.2  public.ecr.aws-*
172.17.0.3  runner-*

Notice the missing "docker" hostname mapping that's present in standard Docker Hub deployments.

Here are three working solutions to address this ECR authentication issue:

Option 1: Explicit Docker Host Configuration

Override the default Docker host configuration:

build-push:
  stage: package
  image: public.ecr.aws/x/x
  services:
    - name: public.ecr.aws/x/docker-dind:20.10
      alias: docker
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""

Option 2: Using Docker Socket Mount

Alternative approach without DinD:

build-push:
  stage: package
  image: docker:20.10
  variables:
    DOCKER_HOST: unix:///var/run/docker.sock
  services:
    - docker:20.10-dind

Option 3: Custom DNS Configuration

For advanced cases where you need to maintain DinD:

before_script:
  - echo "172.17.0.1 docker" >> /etc/hosts
  - docker --host tcp://docker:2375 info

When troubleshooting similar issues:

  • Always check container networking with docker network inspect
  • Verify DNS resolution inside containers
  • Compare environment variables between working and failing setups