Fix “rcmd: socket: Permission denied” Error in pdsh When Running Remote Commands


2 views

I recently encountered this frustrating error while trying to execute remote commands across multiple servers using pdsh:

pdsh -w host1,host2 tail -f /var/log/apache.log
pdsh@myhost: host2: rcmd: socket: Permission denied
pdsh@myhost: host1: rcmd: socket: Permission denied

After digging through documentation and testing various scenarios, I found these are the most common root causes:

  • SSH key authentication not properly configured
  • Firewall blocking the connection
  • Incorrect permissions on ~/.ssh directory
  • pdsh not using the correct authentication method

Here's what worked for me:

# First, ensure passwordless SSH works
ssh host1 "tail -f /var/log/apache.log"

# If that works, check pdsh configuration
export PDSH_RCMD_TYPE=ssh
pdsh -w host1,host2 tail -f /var/log/apache.log

For more complex environments, you might need to modify your ~/.pdshrc file:

# Sample .pdshrc configuration
PDSH_SSH_ARGS_APPEND="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
PDSH_RCMD_TYPE=ssh

If you're still facing issues, try these diagnostic commands:

# Check SSH connectivity
pdsh -w host1,host2 -S -l username uptime

# Verify permissions
ls -ld ~/.ssh
chmod 700 ~/.ssh
chmod 600 ~/.ssh/*

If you're stuck, consider using parallel-ssh as an alternative:

pssh -H "host1 host2" -i "tail -f /var/log/apache.log"

When executing parallel commands across multiple hosts using pdsh, the "rcmd: socket: Permission denied" error typically indicates a fundamental connectivity or authentication issue. This commonly occurs when:

  • The initiating user lacks proper SSH permissions
  • Firewall rules block the required ports
  • SSH keys aren't properly configured
  • The remote hosts deny the connection attempt

First verify basic SSH connectivity:

ssh host1 "tail -f /var/log/apache.log"
ssh host2 "tail -f /var/log/apache.log"

If these work but pdsh fails, we're likely dealing with either:

  1. pdsh configuration issues
  2. SSH agent forwarding problems
  3. Permission restrictions in /etc/hosts.allow or /etc/hosts.deny

1. Explicitly Specify SSH Module

Force pdsh to use SSH:

pdsh -R ssh -w host1,host2 tail -f /var/log/apache.log

2. Configure SSH Keys Properly

Ensure your public key exists in ~/.ssh/authorized_keys on all target hosts:

ssh-copy-id -i ~/.ssh/id_rsa.pub user@host1
ssh-copy-id -i ~/.ssh/id_rsa.pub user@host2

3. Check Remote Shell Configuration

Some systems restrict remote command execution. Verify in /etc/ssh/sshd_config:

PermitUserEnvironment yes
AllowAgentForwarding yes

Increase verbosity to identify the failure point:

PDSH_SSH_ARGS="-v" pdsh -w host1,host2 echo test

For systems using SELinux, check audit logs:

grep pdsh /var/log/audit/audit.log | audit2allow

If pdsh continues to fail, consider these alternatives:

# Using GNU parallel
parallel --nonall --slf hostlist.txt "tail -f /var/log/apache.log"

# Using clustershell
clush -w host1,host2 "tail -f /var/log/apache.log"