When testing GELF message formats with echo commands, you might encounter silent failures where:

• Messages don't appear in Graylog despite active connections
• Only server restart messages are visible
• Debug logs show zero traces of incoming messages

First verify your input configuration matches these requirements:

# Example working GELF TCP input configuration (graylog.conf)
input {
  gelf_tcp {
    port = 12201
    bind_address = 0.0.0.0
    recv_buffer_size = 1MB
  }
}

Instead of basic echo, use these more reliable testing approaches:

Method 1: Using netcat with proper termination

echo -n '{
  "version": "1.1",
  "host": "test.example",
  "short_message": "GELF test",
  "level": 5,
  "_test_uid": 42,
  "_type": "gelf_tcp_test"
}' | nc -q 1 graylog.example.com 12201

Key differences from original approach:

• -n flag prevents newline issues
• -q 1 ensures proper connection closing (Linux)
• Simplified JSON structure for better parsing

Method 2: Python validation script

import socket
import json

def send_gelf_tcp(message, host='graylog.example.com', port=12201):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
        sock.connect((host, port))
        sock.sendall(json.dumps(message).encode('utf-8'))
        sock.send(b'\x00')  # Null byte termination
    finally:
        sock.close()

test_msg = {
    "version": "1.1",
    "host": "python-sender",
    "short_message": "TCP validation test",
    "level": 3,
    "_protocol": "gelf_tcp",
    "_facility": "python-test"
}

send_gelf_tcp(test_msg)

Network-level verification

# Check if port is actually listening
telnet graylog.example.com 12201
# Or:
nc -zv graylog.example.com 12201

# Packet capture test (requires sudo)
tcpdump -i any 'port 12201' -w gelf_test.pcap

Graylog-side diagnostics

Enable these settings in your Graylog node:

# In graylog.conf
debug = true
log_level = DEBUG

# Restart and monitor logs
journalctl -u graylog-server -f | grep -i gelf

• Missing null byte: GELF/TCP expects null-terminated messages
• JSON formatting: Newlines in echo commands can corrupt payloads
• Buffer sizes: Large messages may be silently dropped
• Connection persistence: TCP connections staying open too long

If standard methods fail, try these alternatives:

# Using curl for HTTP GELF (different port)
curl -X POST http://graylog:12202/gelf \
  -H "Content-Type: application/json" \
  -d '{"version":"1.1","host":"curl-test","short_message":"HTTP test"}'

Remember that GELF TCP and UDP have different requirements than HTTP transport.

---

When testing Graylog's GELF TCP input on port 12201, many developers encounter a frustrating scenario where echo messages vanish into the void. The server shows active connections but no message content appears in the logs, even with --debug enabled.

First, verify your input configuration with these essentials:

# Example input configuration (from graylog.conf)
input {
  gelf_tcp {
    port = 12201
    bind_address = 0.0.0.0
    recv_buffer_size = 1MB
    tcp_keepalive = true
  }
}

Key parameters to check:

• Firewall rules allowing traffic on port 12201
• Correct bind_address (0.0.0.0 for all interfaces)
• Sufficient recv_buffer_size for message volume

Your message format looks correct, but let's examine a more robust version:

echo -n '{
  "version": "1.1",
  "host": "example.org",
  "short_message": "Test message",
  "full_message": "Detailed\\nmultiline\\ncontent",
  "timestamp": '$(date +%s.%3N)',
  "level": 6,
  "_app": "test-script",
  "_env": "staging"
}' | nc -w 2 -N logging.example.com 12201

Critical improvements:

• -n flag prevents newline corruption
• -N forces immediate close after send
• Explicit timestamp avoids parsing issues
• Additional metadata fields help debugging

When the application layer fails, go lower:

# Verify port accessibility
telnet graylog.example.com 12201
# Or with netcat:
nc -zv graylog.example.com 12201

# Check firewall rules (Linux)
sudo iptables -L -n | grep 12201

# Capture network traffic
sudo tcpdump -i any port 12201 -A -vv

Enable deeper logging in Graylog:

# Edit graylog-server.conf
debug_log = true
log_level = DEBUG

# Then monitor the logs
tail -f /var/log/graylog-server/server.log | grep -i gelf

Check these diagnostic endpoints:

• /system/inputs - Verify input status
• /system/metrics - Inspect incoming message counters
• /system/threaddump - Check for blocked threads

When echo and nc prove unreliable, try these:

Python test script:

import socket
import json
import time

gelf_msg = {
    "version": "1.1",
    "host": socket.gethostname(),
    "short_message": "Python test",
    "timestamp": time.time(),
    "level": 1
}

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('graylog.example.com', 12201))
sock.sendall(json.dumps(gelf_msg).encode('utf-8'))
sock.close()

Using curl (for HTTP-input-configured Graylog):

curl -X POST -H "Content-Type: application/json" \
-d '{"version":"1.1","host":"curl-test","short_message":"Curl test"}' \
http://graylog.example.com:12201/gelf