When debugging applications that communicate via HTTP/XML on localhost, traditional network monitoring tools might not show the complete picture. The core issue lies in capturing the actual payload being transmitted, especially when dealing with compressed (.gz) or XML-formatted data.

sudo apt-get install tcpdump wireshark

To capture HTTP traffic on a specific port (e.g., 8080):

sudo tcpdump -i lo -A -s 0 'tcp port 8080 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

When specifically looking for XML traffic:

sudo tcpdump -i lo -A -s 0 'tcp port 8080 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x3c3f786d)'

For GUI-based analysis:

1. Start Wireshark with sudo wireshark
2. Select the loopback interface (lo)
3. Apply filter: tcp.port == 8080 && http
4. Right-click → Follow → TCP Stream

To automatically decompress captured gzip content:

sudo tcpdump -i lo -w capture.pcap
# Then in Wireshark:
# Edit → Preferences → Protocols → HTTP → Decompress entity bodies

Sample command to capture XML-RPC style communication:

sudo tcpdump -i lo -A -s 0 'tcp port 8080 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x3c3f786d)' | grep -A50 "<methodCall>"

For script-based processing:

tshark -r capture.pcap -Y "http and tcp.port == 8080" -T json

Remember that capturing localhost traffic requires root privileges. Always:

• Use temporary privilege elevation with sudo
• Limit capture duration
• Store captures in protected directories

If you're not seeing the expected content:

1. Verify the correct port: netstat -tulnp | grep LISTEN
2. Check for TLS encryption (you'll need to decrypt if present)
3. Confirm the interface (use ifconfig to verify 'lo')

When debugging applications with embedded HTTP servers (especially those using compressed formats like .gz), developers often need visibility into the actual XML payloads being exchanged. Traditional packet sniffers might show connection establishment but fail to display the application-layer content.

While tcpdump is powerful, we'll explore a more targeted approach combining multiple tools:

# Basic tcpdump command to capture port traffic
sudo tcpdump -i lo -A -s 0 port 8080 -w capture.pcap

For applications using gzip compression (common with .gz extensions), consider this processing pipeline:

# 1. Capture traffic
tcpdump -i lo -w raw.pcap port 1234

# 2. Extract HTTP streams
tshark -r raw.pcap -Y "http" -T json > http_streams.json

# 3. Decompress if needed
for stream in $(jq '.[]._source.layers.http' http_streams.json); do
  echo $stream | base64 -d | gunzip > decompressed.xml
done

When you need real-time monitoring without post-processing:

sudo ngrep -d lo -W byline port 8080 and '(POST|GET)'

Here's a complete workflow to extract XML from localhost traffic:

# Step 1: Identify the target port
netstat -tulnp | grep -i application_name

# Step 2: Launch capture with verbose output
sudo tcpdump -i lo -A -s 0 'tcp port 3000 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' 

# Step 3: Filter XML content
tcpdump -i lo -A -s 0 port 3000 | grep -E '<\\?xml|<[a-zA-Z]+>' --color=auto

For GUI-based inspection, export your capture and use Wireshark's HTTP filters:

1. File → Import from Hex Dump
2. Apply filter: http.content_type contains "xml"
3. Right-click → Follow → HTTP Stream

If the embedded server uses HTTPS, you'll need to configure SSL decryption:

# Method 1: MITM with pre-master secret logging
export SSLKEYLOGFILE=~/sslkeylog.txt
# Then configure client to use this (if possible)

# Method 2: Decrypt in Wireshark using server private key
Edit → Preferences → Protocols → TLS → Add RSA key