Production-Ready Multi-Master PostgreSQL Replication Solutions: Evaluating XC, pgpool-II and Alternatives


3 views

When building distributed PostgreSQL systems, true multi-master replication remains one of the most challenging requirements. Let's examine the current state of available solutions:


// Typical multi-master setup requirements
const requirements = {
  readScaling: true,
  writeScaling: true,
  conflictResolution: 'required',
  latency: '<100ms',
  sqlCompatibility: 'full'
};

The Postgres-XC project shows promise but has significant gaps in SQL implementation. For example:


CREATE TABLE users (
  id SERIAL PRIMARY KEY,  -- Fails in XC as of version 2.0
  username VARCHAR(50)
);

The missing SERIAL type support makes migration challenging for existing applications.

pgpool-II remains the most battle-tested solution currently. Here's a basic 2-node configuration:


# pgpool.conf fragment
backend_hostname0 = 'primary1.example.com'
backend_port0 = 5432
backend_weight0 = 1
backend_flag0 = 'ALLOW_TO_FAILOVER'

backend_hostname1 = 'primary2.example.com'
backend_port1 = 5432
backend_weight1 = 1
backend_flag1 = 'ALLOW_TO_FAILOVER'

Several newer projects are worth monitoring:

  • Citus: Sharding-focused but adding multi-master capabilities
  • BDR (Bi-Directional Replication): 2ndQuadrant's commercial offering
  • PostgreSQL + etcd: Custom solutions using consensus protocols

All multi-master systems need conflict handling. A common approach:


-- Timestamp-based resolution example
CREATE FUNCTION resolve_conflict() RETURNS TRIGGER AS $$
BEGIN
  IF NEW.last_modified > OLD.last_modified THEN
    RETURN NEW;
  ELSE
    RETURN OLD;
  END IF;
END;
$$ LANGUAGE plpgsql;

When evaluating solutions, consider:

  • Network partition tolerance
  • WAN latency implications
  • Administrative overhead
  • Monitoring requirements

When evaluating PostgreSQL multi-master replication options, we encounter several solutions with varying maturity levels. Here's my hands-on experience with the major contenders:

// Example connection setup for pgpool-II
# pgpool.conf snippet for multi-master
backend_hostname0 = 'primary1.example.com'
backend_port0 = 5432
backend_weight0 = 1
backend_hostname1 = 'primary2.example.com'
backend_port1 = 5432
backend_weight1 = 1

The most glaring issue I've found with Postgres-XC is its incomplete SQL implementation. Basic features like SERIAL columns aren't fully supported, which breaks many existing applications. During testing, this simple table creation failed:

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    username VARCHAR(50) NOT NULL
);
// Error: SERIAL type not properly implemented in this version

While pgpool-II works well for two-node setups, its multi-master capabilities beyond two nodes become problematic. The load balancing works, but conflict resolution requires manual intervention. Here's a common scenario I've encountered:

-- Node1
BEGIN;
UPDATE accounts SET balance = balance - 100 WHERE user_id = 5;

-- Simultaneously on Node2
BEGIN;
UPDATE accounts SET balance = balance + 200 WHERE user_id = 5;
-- Results in replication conflict requiring DBA resolution

Several newer projects show promise but aren't yet production-ready:

  • Postgres-BDR (Bi-Directional Replication) - Supports conflict resolution but requires PG 9.4+
  • Citus - More focused on sharding than true multi-master
  • Patroni with etcd - Provides HA but not write-anywhere capability

For immediate production needs, I've had success with this architecture:

Application Layer → HAProxy → Multiple pgpool-II instances → PostgreSQL pairs
                      ↑
                  Keepalived for VIP

The key is to limit each pgpool-II instance to managing just two PostgreSQL nodes, then use HAProxy to distribute load across multiple pgpool instances. This maintains write capability across all nodes while avoiding pgpool's scaling limitations.

Implement application-level conflict resolution with these approaches:

-- Example of timestamp-based resolution
CREATE FUNCTION resolve_conflict() RETURNS TRIGGER AS $$
BEGIN
    IF NEW.last_modified > OLD.last_modified THEN
        RETURN NEW;
    ELSE
        RETURN OLD;
    END IF;
END;
$$ LANGUAGE plpgsql;