Troubleshooting Docker CE Network Connectivity Issues on CentOS 8: Published Ports and Outbound Access


1 views

After a fresh Docker CE 19.03.3 installation on CentOS 8.0.1905 with NetworkManager and FirewallD enabled, we're observing two critical network issues:

  • Published container ports aren't accessible from external hosts
  • Containers cannot initiate outbound network connections

First, let's verify the basic network configuration:


# Check Docker bridge network
sudo docker network inspect bridge

# Verify iptables rules
sudo iptables -L -n -v --line-numbers
sudo iptables -t nat -L -n -v --line-numbers

# Test container connectivity
sudo docker run --rm busybox ping -c 4 8.8.8.8

The default CentOS 8 public zone blocks Docker traffic. Here's how to properly configure FirewallD:


# Add Docker interface to trusted zone
sudo firewall-cmd --permanent --zone=trusted --add-interface=docker0

# Allow container traffic through NAT
sudo firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0 -i docker0 -j ACCEPT
sudo firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -i docker0 -j ACCEPT

# Reload firewall configuration
sudo firewall-cmd --reload

Ensure kernel parameters and NAT rules are properly set:


# Enable IP forwarding
echo "net.ipv4.ip_forward=1" | sudo tee /etc/sysctl.d/99-docker.conf
sudo sysctl -p /etc/sysctl.d/99-docker.conf

# Verify NAT masquerading is enabled
sudo firewall-cmd --query-masquerade

Here's a complete test scenario demonstrating proper configuration:


# Run a test web server
sudo docker run -d -p 8080:80 --name web nginx:alpine

# Check port mapping
sudo docker port web

# Test connectivity from host
curl -I localhost:8080

# Test from another host (replace with actual IP)
curl -I your_server_ip:8080

# Test container outbound access
sudo docker exec web curl -I example.com

For development environments only, you might consider:


# Stop and disable FirewallD (not recommended for production)
sudo systemctl stop firewalld
sudo systemctl disable firewalld

# Restart Docker
sudo systemctl restart docker

For production systems, create a permanent FirewallD service definition:


# Create Docker service XML
sudo tee /etc/firewalld/services/docker.xml <<EOF
<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>Docker</short>
  <description>Docker CE network requirements</description>
  <port protocol="tcp" port="2375"/>
  <port protocol="tcp" port="2376"/>
</service>
EOF

# Add service to public zone
sudo firewall-cmd --permanent --zone=public --add-service=docker
sudo firewall-cmd --reload

After a fresh CentOS 8 minimal installation with NetworkManager and FirewallD enabled (default zone set to public), I encountered networking issues with Docker CE 19.03.3:

  • Published ports weren't accessible from other servers
  • Containers couldn't reach external networks
  • Basic ping tests failed both directions

First, verify basic container networking:

# Run a test container
docker run --rm -it centos:8 bash

# Inside container:
ping 8.8.8.8
curl google.com

# On host machine:
docker run -d -p 80:80 nginx
curl localhost  # Should work

CentOS 8's default firewall blocks Docker traffic. Add these rules:

sudo firewall-cmd --permanent --zone=public --add-interface=docker0
sudo firewall-cmd --permanent --zone=public --add-port=2376/tcp
sudo firewall-cmd --permanent --zone=public --add-port=2377/tcp
sudo firewall-cmd --permanent --zone=public --add-port=7946/tcp
sudo firewall-cmd --permanent --zone=public --add-port=7946/udp
sudo firewall-cmd --permanent --zone=public --add-port=4789/udp
sudo firewall-cmd --reload

The main culprit was NetworkManager modifying iptables rules. Create this configuration file:

sudo tee /etc/NetworkManager/conf.d/docker.conf <<EOF
[keyfile]
unmanaged-devices=interface-name:docker0;interface-name:br-*;interface-name:veth*
EOF

sudo systemctl restart NetworkManager

Ensure IP forwarding is enabled in sysctl:

sudo sysctl -w net.ipv4.ip_forward=1
echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.conf

Here's the full remediation process:

  1. Stop Docker service: sudo systemctl stop docker
  2. Flush iptables: sudo iptables -F
  3. Apply NetworkManager configuration
  4. Configure FirewallD rules
  5. Enable IP forwarding
  6. Restart services: sudo systemctl restart NetworkManager docker

For development environments only:

sudo systemctl stop firewalld
sudo systemctl disable firewalld

Remember to test connectivity after each configuration change.

Validate your setup with these diagnostic commands:

# Check NAT rules
sudo iptables -t nat -L -n -v

# Verify bridge status
brctl show

# Inspect Docker networks
docker network inspect bridge

# Check published ports
ss -tulnp | grep docker