When dealing with mass file operations through Samba on Ubuntu, many administrators hit an invisible wall at exactly 1024 open files - despite having configured higher limits through every conventional method. The root cause often lies in deeper system configuration layers that override your settings.
First, let's confirm what limits the Samba process is actually running with:
# Check running smbd process limits
cat /proc/$(pgrep smbd)/limits | grep "Max open files"
# Alternative method using prlimit
prlimit --pid $(pgrep smbd) --nofile
For Ubuntu 10.04 (using Upstart), create or modify /etc/init/smbd.conf
:
limit nofile 50000 65000
pre-start script
ulimit -n 50000
end script
Even with correct limits.conf settings, the PAM module might not apply them to daemon processes. Edit /etc/pam.d/common-session
:
session required pam_limits.so
Add these to your smb.conf
under the [global] section:
max open files = 65535
getwd cache = yes
aio read size = 16384
aio write size = 16384
For systems handling thousands of small files, these sysctl tweaks help:
echo "fs.file-max = 2097152" >> /etc/sysctl.conf
echo "fs.aio-max-nr = 1048576" >> /etc/sysctl.conf
sysctl -p
On the Windows 7 client, adjust these registry settings:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters]
"MaxCmds"=dword:00001000
"MaxThreads"=dword:00001000
"MaxCollectionCount"=dword:00000400
When all else fails, consider these approaches:
- Use
rsync
over SSH instead of SMB for mass small file transfers - Archive files before transfer using
tar
on the Ubuntu side - Implement a custom Python script using
pyfilesystem
for controlled transfers
When transferring thousands of small files through Samba on Ubuntu 10.04, Windows 7 clients hit an unexpected ceiling at approximately 1024 open files despite explicit configuration changes. The server logs clearly indicate Samba recognizes higher limits (50,000 in this case), yet the practical limit remains stubbornly fixed.
Through diagnostic commands like lsof | wc -l
, we observe the hard stop occurs precisely around 1000-1024 files. Key findings from troubleshooting:
# System-wide file descriptor check
cat /proc/sys/fs/file-max
# Process-specific limits
cat /proc/$(pgrep smbd)/limits | grep 'Max open files'
Ubuntu 10.04's upstart init system requires special attention for service-level limits. The complete init configuration should include:
# /etc/init/smbd.conf
limit nofile 25000 65000
pre-start script
ulimit -n 25000
ulimit -a > /var/log/samba/ulimit.log
end script
For Windows clients transferring massive file sets, implement these registry tweaks:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters]
"MaxCmds"=dword:00001000
"MaxThreads"=dword:00001000
Essential smb.conf parameters for high-filecount environments:
[global]
max open files = 50000
getwd cache = yes
socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536
strict locking = no
oplocks = yes
kernel oplocks = no
Add these to /etc/sysctl.conf for better file handling:
fs.file-max = 500000
fs.nr_open = 500000
net.ipv4.tcp_window_scaling = 1
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216