Redis maxclients Reduction Due to Persistent 1024 File Descriptor Limit Despite limits.conf Update


2 views

When Redis encounters this warning message:

Current maximum open files is 1024. maxclients has been reduced to 4064 to compensate for low ulimit

It indicates Redis cannot acquire the necessary file descriptors despite your system configuration. Let's examine the complete diagnostic path.

# Check effective limits for the running process
cat /proc/$(pgrep redis-server)/limits | grep 'Max open files'

# Alternative method using prlimit
sudo prlimit --pid $(pgrep redis-server) --nofile

1. Systemd Service Configuration Override (common on Ubuntu 14.04+):

# Create/edit systemd override
sudo mkdir -p /etc/systemd/system/redis.service.d
sudo nano /etc/systemd/system/redis.service.d/limits.conf

[Service]
LimitNOFILE=65535

2. AppArmor/SELinux Restrictions:

# Check AppArmor status
sudo aa-status
# Temporarily disable for testing
sudo systemctl stop apparmor

For Ubuntu systems using upstart (14.04 default):

# Edit redis upstart config
sudo nano /etc/init/redis.conf

# Add limit stanza before 'exec' line
limit nofile 65535 65535

For systems with pam_limits:

# Ensure pam_limits is loaded
sudo nano /etc/pam.d/common-session
# Add or verify this line exists
session required pam_limits.so

Create a diagnostic script to verify all components:

#!/bin/bash
echo "System-wide limits:"
cat /etc/security/limits.conf | grep -v '^#' | grep -v '^$'

echo -e "\nProcess limits:"
sudo cat /proc/$(pgrep redis-server)/limits | grep 'Max open files'

echo -e "\nSystemd configuration:"
systemctl cat redis 2>/dev/null || echo "No systemd unit found"

echo -e "\nEffective ulimit:"
sudo -u redis bash -c 'ulimit -n'

For high-performance setups requiring >10k connections:

# Kernel parameters
sudo nano /etc/sysctl.conf

fs.file-max = 2097152
fs.nr_open = 2097152
net.core.somaxconn = 32768

Then apply changes:

sudo sysctl -p
sudo service procps start

After implementing all changes:

# Fully restart Redis
sudo service redis-server restart

# Verify in Redis CLI
redis-cli config get maxclients
redis-cli info clients | grep connected_clients

Despite updating /etc/security/limits.conf and verifying with ulimit -n, Redis continues to report:

Current maximum open files is 1024. maxclients has been reduced to 4064 to compensate for low ulimit.
You requested maxclients of 10000 requiring at least 10032 max file descriptors.
Redis can't set maximum open files to 10032 because of OS error: Operation not permitted.

On Ubuntu 14.04+ using systemd, the traditional limits.conf approach doesn't work for services. Redis running under systemd needs separate configuration:

# Create/edit the systemd service override
sudo mkdir -p /etc/systemd/system/redis.service.d
sudo nano /etc/systemd/system/redis.service.d/limits.conf

Add this content:

[Service]
LimitNOFILE=65535
  1. Apply systemd limits:
  2. sudo systemctl daemon-reload
    sudo systemctl restart redis
    
  3. Verify Redis process limits:
  4. cat /proc/$(pgrep redis)/limits | grep files
    Max open files            65535                65535                files
    
  5. Update Redis config:
  6. # In redis.conf
    maxclients 10000
    

For older Ubuntu versions using Upstart:

# Edit /etc/init/redis.conf
limit nofile 65535 65535

Run this to confirm all settings are applied correctly:

#!/bin/bash
echo "System-wide limit: $(sysctl fs.file-max)"
echo "User limit: $(ulimit -n)"
echo "Redis process limit: $(cat /proc/$(pgrep redis)/limits | grep files)"
redis-cli config get maxclients