Resolving OpenVPN Port-Sharing Conflicts with Apache SSL on Port 443


4 views

When attempting to share port 443 between OpenVPN and Apache's HTTPS service using port-share, many administrators encounter protocol detection issues. The fundamental problem manifests when non-OpenVPN HTTPS traffic gets misdirected or improperly handled.

The error ssl_error_rx_record_too_long typically indicates that:

  • OpenVPN is receiving standard HTTPS traffic it cannot process
  • The port-sharing handoff isn't properly recognizing protocol differences
  • Apache is receiving OpenVPN's binary protocol as HTTP requests

The original setup attempts:

# OpenVPN config
local ${PUBLIC_IP}
port 443
port-share localhost 443

# Apache config
Listen localhost:443

This creates a circular reference where both services compete for the same port on localhost.

Here's a working configuration that maintains security while allowing both services to coexist:

# OpenVPN Server Configuration (server.conf)
port 443
proto tcp-server
port-share 127.0.0.1 10443
dev tun
persist-key
persist-tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
server 10.8.0.0 255.255.255.0
push "redirect-gateway def1 bypass-dhcp"
keepalive 10 120
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status openvpn-status.log
verb 3

And the corresponding Apache configuration:

# Apache SSL Configuration
Listen 127.0.0.1:10443

<VirtualHost 127.0.0.1:10443>
    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/server.crt
    SSLCertificateKeyFile /etc/ssl/private/server.key
    SSLCertificateChainFile /etc/ssl/certs/ca.crt
    # Rest of your configuration
</VirtualHost>

1. Firewall Considerations: Ensure port 443 TCP is open in your firewall rules

2. Protocol Handling: OpenVPN must use proto tcp-server for port-sharing to work

3. Port Selection: The shared port (10443 in this case) should be above 1024 and unused

To verify proper traffic routing:

# Check port listening status
netstat -tulnp | grep -E '443|10443'

# Test OpenVPN connection
openvpn --client --config client.ovpn

# Test HTTPS separately
curl -vk https://yourdomain.com

For environments where port-sharing proves problematic, consider:

  • Running OpenVPN on UDP/443 instead (if UDP is acceptable)
  • Using HAProxy for more sophisticated protocol detection
  • Implementing SNI-based routing with newer OpenVPN versions

When attempting to configure OpenVPN to share port 443 with Apache's HTTPS service using port-share, you might encounter SSL protocol errors like:

Firefox: "SSL received a record that exceeded the maximum permissible length"
Curl: "error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol"

Apache error logs show invalid method requests:

[error] [client 127.0.0.1] Invalid method in request \x16\x03\x01

The fundamental issue is that OpenVPN's port-share feature has limitations:

  • It only works with plain HTTP traffic, not HTTPS
  • The feature was designed for simple HTTP proxy scenarios
  • SSL/TLS handshakes get misinterpreted as OpenVPN protocol packets

Option 1: Separate Ports (Recommended)

The most reliable approach is to use different ports:

# OpenVPN config
port 1194
proto udp

# Apache config
Listen 0.0.0.0:443

Option 2: SSLH - Application Protocol Multiplexer

For true port sharing, use sslh:

# Install sslh
sudo apt-get install sslh

# Configuration example (/etc/sslh.cfg):
{
    "listen": "0.0.0.0:443",
    "protocols": [
        { "name": "openvpn", "service": "openvpn", "host": "localhost", "port": "1194" },
        { "name": "https", "service": "http", "host": "localhost", "port": "443" }
    ]
}

Option 3: Nginx Reverse Proxy

If you must use port 443 for both:

# nginx configuration
server {
    listen 443 ssl;
    server_name yourdomain.com;
    
    # SSL configuration here
    
    location / {
        proxy_pass http://localhost:8080;
    }
}

stream {
    server {
        listen 443;
        proxy_pass localhost:1194;
    }
}

If you still want to experiment with port-share:

  1. Verify OpenVPN version supports the feature
  2. Check for port conflicts with netstat -tulnp | grep 443
  3. Test with plain HTTP first before attempting HTTPS

Remember that OpenVPN's port-share was never designed to handle modern SSL/TLS traffic properly. The solutions above provide more robust alternatives.