When examining the log entries, we observe a critical sequence of errors:

Jan 16 21:14:35 steelhorse postfix/smtpd[18426]: lost connection after STARTTLS from localhost.localdomain[127.0.0.1]
Jan 16 21:14:35 steelhorse postfix/smtpd[18426]: warning: TLS library problem: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca:s3_pkt.c:1293:SSL alert number

This indicates the SSL/TLS handshake fails during the authentication phase. The key components involved are:

• Postfix SMTP server configuration
• Dovecot SASL authentication
• Roundcube SMTP client settings
• Certificate chain validation

The self-signed certificate with CN=localhost creates several potential issues:

smtpd_tls_cert_file = /etc/ssl/private/server.crt
smtpd_tls_key_file = /etc/ssl/private/server.key

For proper TLS verification, consider either:

1. Obtaining a valid certificate from Let's Encrypt
2. Creating a proper self-signed certificate with SANs

Example OpenSSL command for proper self-signed cert:

openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes \
  -keyout server.key -out server.crt -subj "/CN=mail.example.com" \
  -addext "subjectAltName=DNS:mail.example.com,DNS:localhost,IP:127.0.0.1"

The critical error in updated logs reveals:

warning: localhost.localdomain[127.0.0.1]: SASL LOGIN authentication failed: Invalid authentication mechanism

Dovecot reports using only PLAIN mechanism:

auth_mechanisms = plain

Yet Roundcube attempts LOGIN mechanism. The solution involves configuring both ends consistently:

In Dovecot (/etc/dovecot/conf.d/10-auth.conf):

auth_mechanisms = plain login

In Postfix (/etc/postfix/main.cf):

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
broken_sasl_auth_clients = yes

The temporary workaround of disabling certificate verification isn't ideal. Better approach:

$config['smtp_conn_options'] = [
    'ssl' => [
        'verify_peer' => true,
        'verify_peer_name' => true,
        'allow_self_signed' => true,
        'verify_depth' => 5,
        'cafile' => '/path/to/ca-certificates.crt',
    ],
];

Key adjustments needed:

• Properly configure the certificate chain
• Ensure the hostname matches the certificate
• Set correct SMTP server address in Roundcube

1. Verify SMTP authentication manually:

openssl s_client -connect localhost:587 -starttls smtp -crlf
EHLO localhost
AUTH PLAIN base64-encoded-credentials

2. Check Dovecot SASL socket permissions:

ls -la /var/run/dovecot/auth-client

3. Validate Postfix to Dovecot communication:

telnet localhost 12345 (dovecot auth port)

Essential items to verify:

Component	Setting	Expected Value
Postfix	smtpd_tls_security_level	may (not encrypt)
Dovecot	auth_mechanisms	plain login
Roundcube	smtp_server	tls://mail.yourdomain.com
Certificates	Subject Alternative Names	Must include all used hostnames

---

The error chain begins with Roundcube failing to authenticate via SMTP, specifically during the TLS handshake phase. The Postfix logs reveal two critical indicators:

Jan 16 21:14:35 steelhorse postfix/smtpd[18426]: lost connection after STARTTLS
Jan 16 21:14:35 steelhorse postfix/smtpd[18426]: warning: TLS library problem: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca:s3_pkt.c:1293:SSL alert number

The most immediate suspect is the SSL certificate configuration. When using self-signed certificates (especially with CN=localhost), you'll need proper certificate chain verification handling:

# For testing purposes, you can temporarily bypass verification in Roundcube:
$config['smtp_conn_options'] = array(
  'ssl' => array(
     'verify_peer' => false,
     'verify_peer_name' => false,
  ),
);

However, for production environments, consider these certificate best practices:

# Proper Postfix TLS configuration in main.cf
smtpd_tls_cert_file=/etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level=may
smtpd_tls_loglevel=1

The subsequent error (535 Authentication Failed) reveals a SASL mechanism mismatch:

Jan 18 20:18:21 steelhorse postfix/smtpd[1942]: warning: SASL LOGIN authentication failed: Invalid authentication mechanism

Ensure Dovecot and Postfix are configured for compatible mechanisms:

# In Dovecot (10-auth.conf):
auth_mechanisms = plain login

# In Postfix main.cf:
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous

To understand the complete authentication flow, enable verbose logging:

# In Postfix main.cf:
debug_peer_level = 2
debug_peer_list = 127.0.0.1

# In Roundcube config.inc.php:
$config['smtp_debug'] = true;

Key things to verify in the debug output:

• The EHLO response showing STARTTLS capability
• Successful TLS negotiation after STARTTLS command
• SASL mechanism negotiation
• Actual authentication attempt

Here's a verified working configuration for the entire stack:

# Postfix main.cf (relevant portions)
smtpd_tls_cert_file=/etc/ssl/certs/mail.crt
smtpd_tls_key_file=/etc/ssl/private/mail.key
smtpd_use_tls=yes
smtpd_tls_auth_only=yes
smtpd_sasl_type=dovecot
smtpd_sasl_path=private/auth
smtpd_sasl_auth_enable=yes
smtpd_sasl_security_options=noanonymous,noplaintext
smtpd_sasl_tls_security_options=noanonymous

# Dovecot 10-auth.conf
auth_mechanisms = plain login
auth_username_format = %n

# Roundcube config.inc.php
$config['smtp_server'] = 'tls://localhost';
$config['smtp_port'] = 587;
$config['smtp_user'] = '%u';
$config['smtp_pass'] = '%p';
$config['smtp_auth_type'] = 'LOGIN';
$config['smtp_conn_options'] = array(
  'ssl' => array(
    'verify_peer' => true,
    'verify_peer_name' => true,
    'allow_self_signed' => false,
  ),
);