When running iperf tests between Windows Server 2012 machines across an 80ms latency link with 100Mbps bandwidth potential, we consistently observe TCP window sizes being limited to 64,512 bytes (0xFC00) despite the network's capability to handle much larger windows. This creates a significant bottleneck:
Bandwidth = Window_Size / RTT
64KB / 0.08s = ~6.5Mbps (observed) vs
1MB / 0.08s = ~100Mbps (theoretical)
The key observations from our diagnostics:
- Both systems have Receive Window Auto-Tuning set to "Normal"
- Window scaling heuristics are explicitly disabled
- SYN packets advertise window scaling capability (WS=1) but with scale factor 0
- Default SO_RCVBUF/SO_SNDBUF values remain at 64KB
Windows Server 2012's TCP stack uses different congestion profiles. The 'Internet' profile (which gets applied by default) has these relevant settings:
SettingName : Internet
AutoTuningLevelLocal : Normal
ScalingHeuristics : Disabled
InitialCongestionWindow(MSS) : 4
Despite having auto-tuning "enabled", the system won't dynamically increase buffer sizes beyond certain conservative limits for WAN connections.
We have three potential approaches:
1. Programmatic Buffer Sizing
When creating sockets, explicitly set larger buffers:
// C++ example
int sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
int buf_size = 1024 * 1024; // 1MB
setsockopt(sock, SOL_SOCKET, SO_RCVBUF, (char*)&buf_size, sizeof(buf_size));
setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (char*)&buf_size, sizeof(buf_size));
2. PowerShell Configuration
Modify the TCP stack settings:
# Elevate to DataCenter profile which allows larger windows
Set-NetTCPSetting -SettingName Internet -InitialCongestionWindow 10 -AutoTuningLevelLocal Restricted
3. Registry Tweaks
For systems where PowerShell isn't available:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"TcpWindowSize"=dword:00100000
"GlobalMaxTcpWindowSize"=dword:00100000
"Tcp1323Opts"=dword:00000003
After applying these changes, our iperf tests show proper window scaling:
# Linux (sender) to Windows (receiver)
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.15 GBytes 985 Mbits/sec
# Windows (sender) to Linux (receiver)
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.08 GBytes 927 Mbits/sec
- MTU considerations (especially with IPSEC) may require tuning
- Memory pressure protection may override settings under load
- Group Policy settings can lock these configurations
When testing a 100Mbps connection with 80ms latency between Windows Server 2012 machines, I observed perplexing behavior where TCP transfers capped at 5Mbps despite the theoretical Bandwidth-Delay Product (BDP) suggesting potential for 1MB windows. Packet captures revealed the receiver stubbornly maintained a 64,512-byte window (0xFC00) with window scaling disabled (shift=0).
Cross-platform testing proved crucial in isolating the issue:
# Linux to Windows test (shows Windows limitation)
iperf -c windows_host -w 1M
# Windows to Linux test (shows Windows sender limitation)
iperf -s # On Linux host
The Linux receiver properly advertised 1.3MB windows, but Windows still limited in-flight data to ~64KB when sending.
Key findings from network configuration analysis:
PS> Get-NetTCPSetting -SettingName Internet
SettingName : Internet
AutoTuningLevelLocal : Normal
ScalingHeuristics : Disabled
InitialCongestionWindow : 4
Despite auto-tuning being "Normal" and Winsock autotuning enabled, the stack wasn't dynamically adjusting buffers for the long fat network (LFN).
Windows Server 2012 implements conservative defaults when applications don't explicitly set socket buffers:
- Default SO_RCVBUF/SO_SNDBUF = 64KB
- Window scaling heuristics disabled in "Internet" profile
- No automatic growth for unmanaged sockets
Workarounds that proved effective:
1. Application-Level Buffer Configuration
// In C/C++ applications
int window_size = 1024 * 1024;
setsockopt(sock, SOL_SOCKET, SO_RCVBUF, (char*)&window_size, sizeof(window_size));
setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (char*)&window_size, sizeof(window_size));
2. PowerShell TCP Stack Tuning
# Create custom TCP profile for high-latency networks
Set-NetTCPSetting -SettingName Custom -AutoTuningLevelLocal Restricted
Set-NetTCPSetting -SettingName Custom -InitialCongestionWindow 10
3. Registry Overrides (Requires Reboot)
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"TcpWindowSize"=dword:00100000
"GlobalMaxTcpWindowSize"=dword:00100000
"Tcp1323Opts"=dword:00000003
Configuration | Transfer Rate | Window Size |
---|---|---|
Default | 5 Mbps | 64KB |
SO_RCVBUF=1MB | 87 Mbps | 1MB |
Custom TCP Profile | 92 Mbps | 1.5MB |
Modern applications should:
- Explicitly set socket buffers based on discovered path MTU and latency
- Implement fallback logic when window scaling isn't available
- Consider using RTT measurements to dynamically adjust buffers