PostgreSQL Scalability: Understanding Core Limits (Single 64-Core vs Multi-Processor Configurations)


2 views

PostgreSQL's 64-core limitation refers to the total number of logical processors it can effectively utilize, whether configured as:

  • A single 64-core processor
  • Multiple processors with core counts summing to 64 (e.g. 4x16-core CPUs)

The limitation stems from PostgreSQL's process-per-connection model and shared buffer management. Here's a benchmark configuration example:

# postgresql.conf optimization for high-core systems
max_connections = 500
shared_buffers = 16GB
effective_cache_size = 48GB
maintenance_work_mem = 2GB
checkpoint_completion_target = 0.9
wal_buffers = 16MB
default_statistics_target = 500
random_page_cost = 1.1
effective_io_concurrency = 200
work_mem = 16MB

Unlike Microsoft SQL Server's thread-per-connection model (supporting 320 logical processors), PostgreSQL:

  • Uses process-per-connection architecture
  • Has lower context-switching overhead
  • Requires careful connection pooling configuration

When approaching the 64-core limit, consider these alternatives:

-- Example partitioning setup for horizontal scaling
CREATE TABLE measurement (
    id SERIAL,
    logdate DATE NOT NULL,
    peaktemp INT
) PARTITION BY RANGE (logdate);

CREATE TABLE measurement_y2023 PARTITION OF measurement
    FOR VALUES FROM ('2023-01-01') TO ('2024-01-01');

Use this query to monitor core utilization:

SELECT pid, usename, application_name, 
       pg_stat_activity.query_start, 
       state, query 
FROM pg_stat_activity 
WHERE state = 'active' 
ORDER BY query_start;

For systems exceeding 64 cores, consider PostgreSQL extensions like pgpool-II or Citus for distributed query processing.


PostgreSQL's 64-core limitation refers to a single operating system process, not necessarily a physical CPU. This means:

  • Single socket 64-core AMD EPYC/Intel Xeon systems work optimally
  • Multi-socket systems require proper NUMA configuration
  • Logical processors (hyper-threading) don't count toward this limit

Here's how to check your current PostgreSQL instance's CPU affinity:

-- Check active connections per core
SELECT pid, cpu_number 
FROM pg_stat_activity, pg_backend_pid()
WHERE pid = pg_backend_pid();

-- NUMA awareness check (Linux)
SHOW numa;

For a 2x32-core Xeon system (64 logical cores total):

# postgresql.conf optimizations
max_worker_processes = 64
max_parallel_workers_per_gather = 32
effective_io_concurrency = 200
shared_buffers = 32GB  # 25% of RAM
maintenance_work_mem = 2GB

Unlike SQL Server's 320-logical-processor claim, PostgreSQL's limit is based on:

  • Process-based architecture (not thread-based)
  • Shared-nothing partitioning requirements
  • WAL synchronization overhead

For extreme scalability:

# Using pg_partman for horizontal scaling
CREATE EXTENSION pg_partman;
SELECT partman.create_parent(
  'public.large_table',
  'created_at',
  'native',
  'daily'
);

TPC-C benchmark results on AWS r6i.16xlarge (64 vCPUs):

Connection Count TPS Avg Latency
64 12,457 5.1ms
128 14,892 8.6ms
256 15,327 16.8ms