Debugging Docker TLS Handshake Timeout: Solutions for “net/http: TLS handshake timeout” Error


2 views

When running docker pull commands on Ubuntu 16.04, many developers encounter this frustrating error:

$ docker pull nginx
Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout

The pcap trace reveals the connection establishes the TCP three-way handshake successfully, but fails during TLS negotiation. What's particularly interesting is that other tools like curl and custom Go programs can connect to the same endpoint:

$ curl https://registry-1.docker.io/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}

Docker's networking stack differs from standard applications in several ways:

  • Uses its own DNS resolution mechanism
  • May route through different network interfaces
  • Has different TLS timeout settings than system defaults

Here are the most effective solutions we've found working with production systems:

# Solution 1: Configure Docker daemon DNS
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <
# Solution 2: Adjust system TCP parameters
echo "net.ipv4.tcp_keepalive_time = 60" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_keepalive_intvl = 10" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_keepalive_probes = 6" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

To better understand the issue, we can mimic Docker's behavior with this Go test program:

package main

import (
    "crypto/tls"
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

func main() {
    transport := &http.Transport{
        TLSHandshakeTimeout: 10 * time.Second,
        DisableKeepAlives:   false,
    }
    
    client := &http.Client{
        Transport: transport,
        Timeout:   30 * time.Second,
    }

    resp, err := client.Get("https://registry-1.docker.io/v2/")
    if err != nil {
        fmt.Printf("Error: %v\n", err)
        return
    }
    defer resp.Body.Close()
    
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Printf("Response: %s\n", body)
}

For enterprise environments with strict firewalls, consider these additional measures:

  1. Configure Docker to use HTTP proxy if needed
  2. Verify MTU settings match your network infrastructure
  3. Check for any transparent proxies that might interfere with TLS

When running docker pull nginx on Ubuntu 16.04 LTS, we consistently encounter:

Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout

The interesting observation is that other HTTPS requests work perfectly:

curl https://registry-1.docker.io/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}

Even a simple Go program mimicking Docker's request succeeds:

package main
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
func main() {
    resp, err := http.Get("https://registry-1.docker.io/v2/")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    fmt.Println("Got: ", string(body))
}

The pcap shows Docker's TLS handshake attempts timing out:

00:38:54.782452 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [S], seq 26945613, win 29200
00:38:54.878630 IP registry-1.docker.io.https > my-ubuntu.52036: Flags [S.], seq 2700732154
[multiple retransmissions observed...]

Based on the symptoms and packet analysis, potential issues include:

  • Docker's MTU settings being too large for the network path
  • Corporate firewall/proxy interfering with Docker's specific TLS handshake
  • System time synchronization issues
  • DNS resolution problems specific to Docker's resolver

1. Verify Docker MTU Settings:

docker network inspect bridge | grep MTU

2. Check System Clock Synchronization:

timedatectl status

3. Test with Different DNS Servers:

docker run --dns 8.8.8.8 --rm nginx

4. Configure Docker Daemon Debug Logging:

sudo nano /etc/docker/daemon.json
{
  "debug": true,
  "log-level": "debug"
}
sudo systemctl restart docker

As a temporary workaround, you can try:

docker pull --platform linux/amd64 nginx

Or configure mirror registry in daemon.json:

{
  "registry-mirrors": ["https://mirror.gcr.io"]
}

For Ubuntu 16.04, consider upgrading Docker:

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io