How to Configure Nginx DNS Resolver for Kubernetes Services in GKE


2 views

When migrating from Docker Compose to Kubernetes (specifically GKE), one common pain point is DNS resolution in Nginx configurations. The working Docker Compose setup:

location /app/ {
    proxy_pass http://webapp:3000/;
    resolver 127.0.0.11; # Docker's internal DNS
}

fails in Kubernetes because the DNS infrastructure is fundamentally different.

In Kubernetes, DNS resolution is handled by kube-dns (or CoreDNS in newer clusters). The DNS server IP is automatically injected into pods via the resolv.conf file. You can verify this by:

kubectl exec -it nginx-pod -- cat /etc/resolv.conf
# Output typically shows something like:
# nameserver 10.0.0.10
# search namespace.svc.cluster.local svc.cluster.local cluster.local

Instead of hardcoding DNS server names, use the Kubernetes-provided DNS IP:

location /app/ {
    proxy_pass http://webapp.namespace.svc.cluster.local:3000/;
    resolver 10.0.0.10 valid=10s; # Use your cluster's DNS IP
    resolver_timeout 5s;
    
    # Standard proxy headers
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}

1. Fully Qualified Domain Name (FQDN): Always use the complete service address service.namespace.svc.cluster.local

2. Dynamic Resolution: The valid parameter controls DNS cache duration

3. Multi-container Pods: For containers in the same pod, use localhost instead:

proxy_pass http://localhost:3000/; # When containers share network namespace

If you still encounter issues:

# Test DNS resolution from the nginx container:
kubectl exec -it nginx-pod -- nslookup webapp

# Check service discovery:
kubectl get endpoints webapp

For production environments, consider adding readiness probes to handle DNS resolution delays during startup.


When moving from Docker Compose to Kubernetes (specifically GKE), many developers encounter DNS resolution issues with Nginx configuration. The main challenge comes from the difference in how DNS resolution works between these environments.

location /app/ {
    proxy_pass http://webapp:3000/;
    resolver kube-dns;  # This won't work
}

Kubernetes has its own DNS service (typically CoreDNS) that runs as a system service. The correct DNS resolver IP isn't a fixed name like "kube-dns" but rather the cluster's DNS service IP.

Here's how to find your cluster's DNS IP:

kubectl get svc -n kube-system | grep kube-dns
# Example output:
# kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP   24d

For a pod with multiple containers (like your nginx+webapp setup), you should use:

location /app/ {
    proxy_pass http://webapp:3000/;
    
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
    
    resolver 10.96.0.10 valid=10s;  # Your cluster's DNS IP
}

If you don't want to hardcode the DNS IP (which might change), consider these approaches:

# Option 1: Use environment variable
resolver $KUBE_DNS_SERVICE_HOST valid=10s;

# Option 2: Use Kubernetes downward API
env KUBE_DNS_SERVICE_HOST;
metadata:
  annotations:
    pod.beta.kubernetes.io/dns-resolution: "true"

If you're still having problems, try these debugging steps:

# Check DNS resolution inside your pod
kubectl exec -it nginx-pod -- nslookup webapp

# Verify CoreDNS logs
kubectl logs -n kube-system -l k8s-app=kube-dns

# Test connectivity between containers
kubectl exec -it nginx-pod -- curl -v http://webapp:3000/health

For production deployments, consider adding these optimizations:

resolver 10.96.0.10 ipv6=off valid=10s;

# Add retry logic for temporary DNS failures
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_intercept_errors on;