Troubleshooting SSL Certificate Chain Validation Errors in POP3 Clients: A Deep Dive into openssl s_client Output Analysis


2 views

When examining SSL/TLS connectivity issues with commands like openssl s_client, we're often confronted with cryptic error messages. The two critical errors in your case are:

verify error:num=20:unable to get local issuer certificate
verify error:num=19:self signed certificate in certificate chain

The fundamental issue lies in certificate chain validation. Let's break down what's happening:

  • The error num=20 indicates OpenSSL cannot find the root CA certificate in its trust store
  • The error num=19 suggests a self-signed certificate exists in the chain
  • Both errors disappear when specifying -CApath or -CAfile

The mono runtime handles SSL certificates differently than standard OpenSSL implementations. Here's how to verify and update mono's certificate store:

# List current certificates
certmgr -list -c Trust
# Add GeoTrust Global CA
mozroots --import --sync --machine
certmgr -add -c Trust /path/to/GeoTrust_Global_CA.cer

Here are concrete approaches to resolve the issue:

Option 1: Bypass Certificate Validation (Not Recommended)

// C# example for Mono
ServicePointManager.ServerCertificateValidationCallback += 
    (sender, cert, chain, sslPolicyErrors) => true;

Option 2: Proper Certificate Pinning

// Better approach: validate specific certificates
ServicePointManager.ServerCertificateValidationCallback +=
    (sender, cert, chain, sslPolicyErrors) =>
    {
        var expectedThumbprint = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
        return cert.GetCertHashString() == expectedThumbprint;
    };

Option 3: Update System Certificates

For Linux systems:

sudo update-ca-certificates

To properly diagnose chain issues:

# Full chain verification
openssl verify -verbose -CApath /etc/ssl/certs cert.pem

# Check certificate dates
openssl x509 -noout -dates -in cert.pem

# Verify against specific CA bundle
openssl s_client -connect secure.emailsrvr.com:995 \
  -CAfile /etc/ssl/certs/ca-certificates.crt

The key difference between the working (Gmail) and problematic (EmailSrvr) chains:

Gmail Chain EmailSrvr Chain
0: pop.gmail.com (Google)
1: Google Internet Authority G2
2: GeoTrust Global CA
3: Equifax Secure CA (root)
0: secure.emailsrvr.com
1: RapidSSL CA
2: GeoTrust Global CA (self-signed)
  1. Update mono's certificate store using mozroots and certmgr
  2. Consider using Let's Encrypt certificates which have better cross-platform support
  3. Implement proper certificate pinning if you control both client and server
  4. Test with different SSL/TLS versions using -ssl3, -tls1, -tls1_1, etc. flags

The core issue manifests when a Mono-based POP3 client fails to connect to secure.emailsrvr.com:995 while other clients work fine. Let's examine the critical differences in OpenSSL output:

# Problematic server (secure.emailsrvr.com)
verify error:num=19:self signed certificate in certificate chain

# Working server (pop.gmail.com) 
verify error:num=20:unable to get local issuer certificate

The key structural difference appears in the certificate chains:

Gmail's Chain:
0: pop.gmail.com → Google Internet Authority G2
1: Google Internet Authority G2 → GeoTrust Global CA
2: GeoTrust Global CA → Equifax Secure CA (external root)

EmailSRVR's Chain:  
0: secure.emailsrvr.com → RapidSSL CA
1: RapidSSL CA → GeoTrust Global CA
2: GeoTrust Global CA → Self-signed (no higher authority)

Mono's certificate validation differs from OpenSSL in several key aspects:

  • More strict handling of self-signed certificates in chains
  • Different default trust store locations
  • Less flexible validation policies

Option 1: Import the intermediate certificates

Create a bundle containing all required certificates:

// C# code to explicitly trust certificates
X509CertificateCollection certificates = new X509CertificateCollection();
certificates.Add(new X509Certificate("GeoTrust_Global_CA.cer"));
certificates.Add(new X509Certificate("RapidSSL_CA.cer"));

ServicePointManager.ServerCertificateValidationCallback += 
    (sender, cert, chain, errors) => {
        // Custom validation logic here
        return true; // Temporary bypass for testing
    };

Option 2: Configure Mono's certificate store

# Linux/macOS command to import certificates
certmgr -ssl https://secure.emailsrvr.com
certmgr -add -c GeoTrust_Global_CA.cer -m Trust

For debugging certificate issues programmatically:

using System;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;

class CertValidator {
    static bool ValidateServerCertificate(
        object sender,
        X509Certificate certificate,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
    {
        Console.WriteLine("Certificate Subject: " + certificate.Subject);
        Console.WriteLine("SSL Policy Errors: " + sslPolicyErrors);
        
        // Inspect the chain
        foreach (var status in chain.ChainStatus)
        {
            Console.WriteLine($"Chain Status: {status.Status} - {status.StatusInformation}");
        }
        
        return sslPolicyErrors == SslPolicyErrors.None;
    }
}

The email provider should consider:

  • Including all intermediate certificates in the server configuration
  • Ensuring the certificate chain terminates at a widely-trusted root
  • Testing with various SSL/TLS implementations (especially Mono)

Recommended commands for troubleshooting:

# Check certificate chain without validation
openssl s_client -connect secure.emailsrvr.com:995 -showcerts -CApath /dev/null

# Verify certificate against system store  
openssl verify -CApath /etc/ssl/certs mail_certificate.pem

# Check Mono's trust store contents
certmgr -list -c -m Trust