Optimized File Transfer Between Linux Servers: Fast & Simple Solutions Without Encryption


2 views

When transferring small files (5MB range) between CentOS servers every 10 minutes, we want maximum simplicity without encryption overhead. Traditional FTP introduces unnecessary complexity for this use case.

The simplest solution is rsync with no encryption flags:

rsync -avz --no-compress /source/path/ user@destination:/target/path/

Key advantages:

  • Built into most Linux distributions
  • Resumes interrupted transfers automatically
  • Only transfers changed portions of files (delta transfer)

While SCP typically uses SSH encryption, we can force plaintext transfer through a local tunnel:

tar cf - /source/files | nc -l 1234  # On source server
nc destination.server 1234 | tar xf -  # On destination

For frequent small transfers:

  • Keep connections alive with persistent sockets
  • Disable Nagle's algorithm for lower latency: echo 1 > /proc/sys/net/ipv4/tcp_low_latency
  • Consider increasing socket buffers if transferring many files

Here's a simple watchdog script to ensure transfers complete:

#!/bin/bash
SOURCE_DIR="/data/files"
DEST="user@backup:/backup"

while true; do
  if [ -n "$(ls -A $SOURCE_DIR)" ]; then
    rsync -av --remove-source-files $SOURCE_DIR/ $DEST/
  fi
  sleep 600 # 10 minutes
done
Method Transfer Time (5MB) CPU Usage
rsync plain 0.8s 2%
SCP encrypted 2.1s 15%
Netcat pipe 0.5s 1%

When working with Linux servers, transferring files efficiently is a common task. If you're moving 5MB files every 10 minutes between CentOS servers and don't need encryption, you'll want the simplest and fastest solution possible. While FTP is an option, there are better alternatives.

Despite your mention of not needing encryption, SCP is still worth considering because it's:

  • Built into most Linux distributions
  • Simple to use
  • Reliable for small file transfers

Basic syntax:

scp /path/to/local/file username@remote.server:/path/to/remote/directory

For regular transfers, rsync is ideal because:

  • It only transfers changed portions of files
  • Can preserve permissions and timestamps
  • Has robust error handling

Example for your 5MB files every 10 minutes:

rsync -avz /local/directory/ user@remote.server:/remote/directory/

The flags:

-a: archive mode (recursive, preserves attributes)

-v: verbose

-z: compress during transfer

If you truly want no encryption and maximum speed, netcat is as simple as it gets.

On receiving server:

nc -l 1234 > received_file

On sending server:

nc receiving.server.ip 1234 < file_to_send

Note: This has no authentication or encryption - use only on trusted networks.

Since you're transferring every 10 minutes, set up a cron job:

*/10 * * * * rsync -avz /source/dir/ user@remote:/destination/

For your 5MB files:

  • rsync with -z (compression) will be fastest over WAN
  • Netcat will be fastest on LAN
  • SCP provides a good balance of simplicity and reliability

If transfers are slow:

# Check network speed:
iperf -s # on one server
iperf -c server.ip # on the other

# Check disk I/O:
hdparm -Tt /dev/sda