Configuring Postfix as a Transparent Email Proxy with Complete Message Archiving for CRM Integration


3 views

When using cloud email services like Google Workspace (formerly G Suite), enterprises often need to implement transparent email archiving solutions that work independently of the primary email provider. The requirement to archive both inbound and outbound messages for CRM integration presents unique technical challenges.

Here's the complete Postfix configuration needed to implement this solution. We'll break it down into logical components:

# /etc/postfix/main.cf
myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = $mydomain
relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_use_tls = yes
smtp_tls_security_level = encrypt
smtp_tls_note_starttls_offer = yes

The key to capturing both directions of email flow lies in using Postfix's content_filter and header_checks mechanisms:

# /etc/postfix/master.cf
smtp      inet  n       -       n       -       -       smtpd
  -o content_filter=archive:

archive   unix  -       n       n       -       -       pipe
  flags=Rq user=archive argv=/usr/local/bin/archive-mail.sh ${sender} ${recipient}

# Capture inbound emails
header_checks = regexp:/etc/postfix/header_checks

Here's a sample archive-mail.sh script that handles both inbound and outbound messages:

#!/bin/bash
# archive-mail.sh

SENDER=$1
RECIPIENT=$2
CRM_DB="crm_archive"
CRM_USER="archive_user"

# Read the entire email from stdin
EMAIL=$(cat -)

# Insert into CRM database
psql -U $CRM_USER -d $CRM_DB -c \
  "INSERT INTO email_archive (sender, recipient, body, direction, timestamp) \
   VALUES ('$SENDER', '$RECIPIENT', '$EMAIL', \
   CASE WHEN '$SENDER' LIKE '%@yourdomain.com' THEN 'outbound' ELSE 'inbound' END, NOW())"

For complete coverage, your DNS MX records should point to your Postfix server which then relays to Google:

yourdomain.com.   3600    IN    MX    10 mail.yourdomain.com.

For high-volume environments, consider these optimizations:

# In main.cf
header_size_limit = 1024000
message_size_limit = 20480000
queue_directory = /var/spool/postfix
default_process_limit = 100

When implementing the database insertion:

  • Use prepared statements to prevent SQL injection
  • Implement proper email parsing to extract metadata
  • Consider async processing for better performance

Essential monitoring commands:

postqueue -p    # View mail queue
postsuper -d ALL    # Clean queue (use with caution)
tail -f /var/log/mail.log    # Monitor in real-time

In enterprise environments where customer communications need to be systematically archived, simply BCC'ing outgoing messages isn't sufficient. We need complete capture of both:

  • All outbound messages sent through Postfix
  • All inbound messages received by Postfix

Here's a comprehensive approach to implement full message retention:

1. Always BCC for Outbound Messages

Add this to main.cf:

always_bcc = archive@yourdomain.com
sender_bcc_maps = hash:/etc/postfix/sender_bcc
recipient_bcc_maps = hash:/etc/postfix/recipient_bcc

2. Content Filter for Inbound Messages

Create a content filter in master.cf:

smtp      inet  n       -       n       -       -       smtpd
  -o content_filter=archive-filter:dummy

Then define the filter service:

archive-filter unix -    n       n       -       -       pipe
  flags=Rq user=archive argv=/usr/local/bin/archive-filter.py
  -o receive_override_options=no_header_body_checks

3. Archive Filter Script

Example Python script (/usr/local/bin/archive-filter.py):

#!/usr/bin/env python
import sys
import email
from email import policy

def main():
    msg = email.message_from_binary_file(sys.stdin.buffer, policy=policy.default)
    # Process message - store in CRM via API
    store_in_crm(msg)

if __name__ == '__main__':
    main()

Message Database Storage

For high-volume environments, consider using Dovecot with LMTP:

archive-dbox unix -      n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver
  -d archive@yourdomain.com -f ${sender} -m userdb_archive

CRM Integration Patterns

When implementing the CRM storage component:

def store_in_crm(msg):
    """Example function to store message in Salesforce"""
    import simple_salesforce
    sf = simple_salesforce.Salesforce(
        username='api_user',
        password='password',
        security_token='token'
    )
    
    sf.EmailMessage.create({
        'Subject': msg['subject'],
        'FromAddress': msg['from'],
        'ToAddress': msg['to'],
        'TextBody': msg.get_body(preferencelist=('plain',)).get_content(),
        'HtmlBody': msg.get_body(preferencelist=('html',)).get_content(),
        'Status': 'Processed'
    })

For high-volume mail servers:

  • Implement queuing for the archive process
  • Consider asynchronous processing
  • Monitor disk I/O for the archive storage