Debugging “rsync getcwd(): No such file or directory” Error in Remote Log Synchronization


8 views

When attempting to synchronize log files between servers using rsync, I encountered a particularly stubborn error:

rsync: getcwd(): No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3)

What's puzzling is that while the identical command works perfectly for Server1, it fails consistently for Server2, despite both being Amazon EC2 instances with identical rsync versions (3.0.6).

Before diving deeper, let's verify some basic checks I performed:

# Verify directory existence on Server2
ssh -p 2188 user@server2 "ls -ld /usr/local/servers/logs/"

# Check working directory behavior
ssh -p 2188 user@server2 pwd

The second command produced the same working directory error, which was a crucial clue.

The error message about getcwd() failing to access parent directories typically indicates one of these scenarios:

  • Permission issues on parent directories
  • Broken symbolic links in the path
  • Filesystem mounting problems
  • Deleted directories still being referenced

After extensive testing, the problem turned out to be related to the shell initialization on Server2. Here's how I fixed it:

# Solution 1: Force a non-interactive shell with full path
rsync -avz -e 'ssh -p 2188' user@server2:/usr/local/servers/logs/* /usr/local/logs/ --rsync-path="/bin/rsync"

# Solution 2: Alternative approach using absolute paths
rsync -avz -e 'ssh -p 2188' user@server2:/usr/local/servers/logs/. /usr/local/logs/ --relative

To avoid similar issues in the future, consider these best practices:

# 1. Always use absolute paths in cron jobs
rsync -avz -e 'ssh -p 2188' user@server2:/full/path/to/logs/ /full/path/to/destination/

# 2. Add environment sanity checks
ssh -p 2188 user@server2 "env > /tmp/rsync_env_check"

# 3. Verify shell initialization
ssh -p 2188 user@server2 "echo $SHELL; test -f ~/.bashrc && echo 'bashrc exists'"

For particularly stubborn cases, these advanced techniques can help:

# Trace system calls
strace -f -e trace=file rsync [options]

# Check filesystem status
ssh -p 2188 user@server2 "df -h; mount | grep usr"

# Verify inode information
ssh -p 2188 user@server2 "stat /usr/local/servers/logs"

The key lesson here is that identical server configurations can still behave differently due to subtle environmental differences. Always check shell initialization and path resolution when encountering getcwd() errors.

When attempting to sync logs from Server2 to LogServer using rsync over SSH with the command:

rsync -avz -e 'ssh -p 2188' user@server2:/usr/local/servers/logs/* /usr/local/logs/

We encounter the following error chain:

shell-init: error retrieving current directory: getcwd: cannot access parent directories: no such file or directory
job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: no such file or directory
rsync: getcwd(): No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at util.c(992) [sender=3.0.6]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [receiver=3.0.6]

First, let's verify directory existence on both sides:

# Check remote directory
ssh -p 2188 user@server2 "ls -ld /usr/local/servers/logs/"

# Check local directory
ls -ld /usr/local/logs/

The error suggests that while the target directory exists, the process cannot properly resolve the current working directory path. This typically happens when:

  • The working directory or one of its parents has been deleted during session
  • Permissions prevent directory traversal
  • The process is in a mount point that became unavailable
  • Symbolic link loops exist in the path

Solution 1: Use Absolute Paths

Modify the rsync command to use absolute paths exclusively:

rsync -avz -e 'ssh -p 2188' --rsync-path='cd / && rsync' \
user@server2:/usr/local/servers/logs/* /usr/local/logs/

Solution 2: Verify Environment Variables

Check if environment variables are causing path resolution issues:

ssh -p 2188 user@server2 "env | grep -E 'PWD|OLDPWD'"

Solution 3: Alternative Directory Navigation

Try changing to root directory first:

rsync -avz -e 'ssh -p 2188' --rsync-path='cd / && rsync' \
user@server2:/usr/local/servers/logs/ /usr/local/logs/

For persistent issues, enable verbose output:

rsync -avvvz -e 'ssh -p 2188' \
user@server2:/usr/local/servers/logs/ /usr/local/logs/

Check process working directory with:

ssh -p 2188 user@server2 "ls -la /proc/self/cwd"

To avoid similar issues:

# Create wrapper script on remote server
cat << 'EOF' | ssh -p 2188 user@server2 "cat > ~/safersync.sh"
#!/bin/bash
cd / || exit 1
exec rsync "$@"
EOF

# Then use it in your rsync command
rsync -avz -e 'ssh -p 2188' --rsync-path='~/safersync.sh' \
user@server2:/usr/local/servers/logs/ /usr/local/logs/