Troubleshooting PostgreSQL SSL Connection Issues with OpenSSL s_client on AWS


3 views

When attempting to establish an SSL connection to a PostgreSQL server on AWS using OpenSSL's s_client (version 0.9.8.k), you might encounter two critical errors:

CONNECTED(00000003)
2036:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:

And in the PostgreSQL server logs:

2009-10-30 13:58:08 UTC LOG:  invalid length of startup packet

This typically indicates a protocol mismatch between the client and server. PostgreSQL expects a proper startup message format, while s_client sends raw SSL handshake data. The "invalid length" error suggests the server is receiving malformed packets during the initial handshake phase.

Instead of using raw s_client, consider these better approaches:

Method 1: Using psql with SSL

psql "host=your-rds-endpoint.rds.amazonaws.com \
      port=5432 \
      dbname=yourdb \
      user=youruser \
      sslmode=verify-full \
      sslrootcert=rds-ca-2019-root.pem"

Method 2: Testing SSL with openssl (correct approach)

First create a test connection:

openssl s_client -connect your-rds-endpoint.rds.amazonaws.com:5432 \
                 -starttls postgres \
                 -showcerts

For AWS RDS PostgreSQL, ensure these settings are in place:

1. Security Group Rules

Inbound Rule:
Type: PostgreSQL
Protocol: TCP
Port Range: 5432
Source: [Your IP] or security group

2. RDS Parameter Group

rds.force_ssl = 1
ssl = 1
  1. Verify network connectivity: telnet your-rds-endpoint 5432
  2. Check PostgreSQL SSL configuration in postgresql.conf:
    ssl = on
    ssl_cert_file = 'server.crt'
    ssl_key_file = 'server.key'
    ssl_ca_file = 'root.crt'
  3. Test with different SSL modes:
    sslmode=disable
    sslmode=allow
    sslmode=prefer
    sslmode=require
    sslmode=verify-ca
    sslmode=verify-full

Here's how to properly establish an SSL connection in Node.js:

const { Client } = require('pg');

const client = new Client({
  user: 'dbuser',
  host: 'your-rds-endpoint.rds.amazonaws.com',
  database: 'mydb',
  password: 'secretpassword',
  port: 5432,
  ssl: {
    rejectUnauthorized: true,
    ca: fs.readFileSync('/path/to/rds-combined-ca-bundle.pem').toString()
  }
});

client.connect()
  .then(() => console.log('Connected via SSL'))
  .catch(e => console.error('Connection error', e.stack));

For AWS RDS, always download the latest certificate bundle:

wget https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

Then specify it in your connection parameters or environment variable:

PGSSLROOTCERT=/path/to/rds-combined-ca-bundle.pem

When attempting to establish an SSL connection to an AWS-hosted PostgreSQL server using OpenSSL's s_client from Windows XP, you're encountering two distinct error messages:

# Client-side error:
CONNECTED(00000003)
2036:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188

# Server-side Postgres log:
2009-10-30 13:58:08 UTC LOG:  invalid length of startup packet

The combination of these errors suggests a protocol mismatch between client and server. With both systems running OpenSSL 0.9.8k (released 2009), we're dealing with legacy encryption standards that may conflict with modern PostgreSQL security requirements.

First, verify the server's SSL configuration in postgresql.conf:

ssl = on
ssl_cert_file = 'server.crt'
ssl_key_file = 'server.key'
ssl_ca_file = 'root.crt'
ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # Legacy-compatible setting

Then test the server's SSL support using this diagnostic command:

openssl s_client -starttls postgres -connect your.aws.host:5432 -showcerts

For legacy systems, you may need to force specific SSL protocols. Try these variations:

# Option 1: Force SSLv3 (if absolutely necessary)
openssl s_client -ssl3 -connect your.aws.host:5432

# Option 2: Modern approach with TLS 1.0
openssl s_client -tls1 -connect your.aws.host:5432

# Option 3: Verbose debugging
openssl s_client -connect your.aws.host:5432 -debug -msg -state

The "invalid length of startup packet" error suggests the SSL handshake is completing, but the subsequent PostgreSQL protocol communication is failing. This often occurs when:

  • The client is sending non-SSL traffic to an SSL-only port
  • The SSL negotiation succeeds but the protocol version is incompatible
  • Packet fragmentation occurs during the handshake

Here's a complete test sequence that should work with legacy systems:

# Generate test packet
(echo -n -e '\0\0\0\x08\x04\xD2\x16\x2F'; sleep 1) | openssl s_client \
-connect your.aws.host:5432 \
-CAfile /path/to/ca.crt \
-cert /path/to/client.crt \
-key /path/to/client.key \
-cipher 'DHE-RSA-AES256-SHA' \
-ssl3

If direct SSL connection proves problematic, consider these approaches:

# Using stunnel as intermediary
[postgresql]
client = yes
accept = 127.0.0.1:5433
connect = your.aws.host:5432
verify = 2
CAfile = /path/to/ca.crt
cert = /path/to/client.crt
key = /path/to/client.key

For AWS RDS PostgreSQL instances, verify these settings:

  1. Security group rules allow inbound traffic on port 5432
  2. The DB parameter group has rds.force_ssl=0 for testing
  3. The certificate bundle is properly configured