In our Windows Server 2008 R2 environment with VMware vSphere 4.0 U1, we observed severe MSMQ performance degradation where:
- 8 web servers (2 vCPUs, 2GB RAM) couldn't push messages beyond 2/sec
- Clustered MSMQ servers (4 vCPUs, 8GB RAM) received only ~7 messages/sec
- Message backlog reached 300K in outbound queues and 1M+ in cluster queues
First, verify these performance counters:
# PowerShell to check MSMQ performance
Get-Counter "\MSMQ Queue(*)\Messages in Queue"
Get-Counter "\MSMQ Service\Messages Received/sec"
Get-Counter "\MSMQ Service\Messages Sent/sec"
Key observations from our environment:
# Sample output from a problematic queue
Path InstanceName CookedValue
---- ------------ -----------
\\SERVER01\MSMQ Queue(webcluster$\processing)\Messages in Queue _total 1048576
For clustered MSMQ environments, apply these registry optimizations:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSMQ\Parameters]
"TCPNoDelay"=dword:00000001
"MessageBufferSize"=dword:00004000
"SocketBufferSize"=dword:00008000
In virtualized environments, pay special attention to:
- VMXNET3 network adapters instead of E1000
- Disable TCP Chimney Offload:
netsh int tcp set global chimney=disabled
- Enable Jumbo Frames on vSwitches (9000 MTU)
When receiving messages programmatically:
// C# optimized message receiver
var queuePath = @".\private$\processing";
using (var queue = new MessageQueue(queuePath))
{
queue.Formatter = new BinaryMessageFormatter();
queue.DefaultPropertiesToSend.UseAuthentication = false;
queue.DefaultPropertiesToSend.UseJournalQueue = false;
// Critical performance settings
queue.SetPermissions("Everyone", MessageQueueAccessRights.FullControl);
queue.MessageReadPropertyFilter.SetAll();
while (true)
{
try
{
// Async peek is crucial for performance
var message = queue.Receive(TimeSpan.FromSeconds(1));
// Process message
}
catch (MessageQueueException mqex)
{
if (mqex.MessageQueueErrorCode != MessageQueueErrorCode.IOTimeout)
throw;
}
}
}
For Windows Server clusters hosting MSMQ:
- Set
HKLM\Cluster\LogLevel
to 3 for detailed logging - Disable SMB signing:
Set-SmbClientConfiguration -RequireSecuritySignature $false
- Adjust CSV cache:
Set-ClusterParameter -Name CsvBlockCacheSize -Value 512
After applying changes, monitor with:
# PowerShell validation script
$counters = @(
"\MSMQ Service\Messages Received/sec",
"\PhysicalDisk(*)\Avg. Disk sec/Transfer",
"\Processor(*)\% Processor Time"
)
Get-Counter -Counter $counters -SampleInterval 5 -MaxSamples 12
In our virtualized infrastructure running vSphere 4.0 Update 1, we encountered severe MSMQ performance degradation with the following characteristics:
// Typical perfmon counters showing the issue
MSMQ Service\\Messages Sent/sec = ~2
MSMQ Service\\Messages Received/sec = ~7
Application Servers\\Messages Processed/sec = 0-12
The environment consists of:
- 8 Windows 2008 Web Edition VMs (2vCPU/2GB RAM) as front-end servers
- 1 Windows 2008 Enterprise Edition clustered MSMQ server (4vCPU/8GB RAM)
- 4 application servers processing messages
When we noticed queues backing up (300K messages in outbound, 1M+ in central queues), we:
- Verified disk space (46GB free of 50GB)
- Upgraded PowerPath to latest version
- Increased cluster nodes from 2vCPU to 4vCPU
- Checked network connectivity between VLANs
Several approaches helped improve throughput:
// PowerShell script to adjust MSMQ performance settings
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\MSMQ\Parameters" -Name "TCPNoDelay" -Value 1
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\MSMQ\Parameters" -Name "MessageDeadletter" -Value 0
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\MSMQ\Parameters" -Name "EnableCache" -Value 1
For clustered MSMQ servers, additional tuning is needed:
- Set HKLM\Cluster\MSMQ\Parameters\EnableTriggerNotification to 0
- Configure HKLM\Cluster\MSMQ\Parameters\ConnectionCacheSize to 5000
- Adjust HKLM\System\CurrentControlSet\Services\MSMQ\Parameters\JournalQuota to 10MB
Virtualization adds complexity to MSMQ performance:
esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1
esxcli system settings advanced set -o /Net/UseHwTSO -i 1
esxcli system settings advanced set -o /Net/TcpipDefAckDelay -i 1
Create a comprehensive monitoring solution with these key counters:
// Sample SCOM or custom monitoring thresholds
MSMQ Queue\\Bytes in Queue > 50MB = Warning
MSMQ Service\\Messages Received/sec < 100 = Critical
MSMQ Queue\\Messages in Queue > 10,000 = Critical