When working with complex network configurations involving multiple routing tables and firewall marks (fwmark), verifying that packets follow the expected path becomes non-trivial. The standard ip route get command doesn't account for packet marks, making it insufficient for testing policy-based routing setups.

After extensive testing, I've found several approaches to properly test fwmark routing:

# Basic syntax with mark (doesn't actually send packets)
ip route get ADDRESS mark MARK

# Example: Check route for 8.8.8.8 with mark 42
ip route get 8.8.8.8 mark 42

For more realistic testing, consider these methods:

# Using socat to create marked packets
socat -u /dev/zero udp:1.2.3.4:1234,setsockopt=1:42:SO_MARK

# Alternative using iptables for marking then testing:
iptables -t mangle -A OUTPUT -d 1.2.3.4 -j MARK --set-mark 42
curl --interface dummy0 http://1.2.3.4

For repeated testing, I created this bash function:

test_fwmark_route() {
    local addr=$1
    local mark=$2
    local dev=${3:-""}
    
    echo "Testing route to $addr with mark $mark"
    ip route get $addr mark $mark ${dev:+dev $dev}
    
    # Optional traceroute variant
    # traceroute -m $mark $addr
}

Key things to check in the output:

• The selected routing table (look for "table TABLE_NAME")
• The outgoing interface (dev IFNAME)
• Next hop (via GATEWAY)
• Any source address selection

Watch out for:

• Missing reverse path filtering (rp_filter) settings
• Conflicting marks between different netfilter chains
• Route caching affecting test results

When the built-in tools aren't enough:

# Using conntrack for connection tracking verification
conntrack -L -m 42

# Network namespace testing (for complex environments)
ip netns exec testns ip route get 8.8.8.8 mark 42

When working with complex routing setups involving firewall marks (fwmark), standard tools like ip route get fall short because they don't account for packet markings. Network engineers need precise ways to test routing decisions that involve:

• Multiple routing tables
• Policy-based routing rules
• iptables/nftables marking

The ip command actually supports fwmark testing through its advanced syntax:

ip route get ADDRESS [from ADDRESS iif DEVICE] \
    [oif DEVICE] [tos TOS] [fwmark MARK] \
    [uid UID] [ipproto PROTOCOL]

Example testing route for marked HTTPS traffic:

# Verify route for packets marked 0x100 destined to 8.8.8.8
ip route get 8.8.8.8 fwmark 0x100

# Full simulation including incoming interface
ip route get 8.8.8.8 from 192.168.1.100 iif eth0 fwmark 0x100

For more complex simulations, create test packets with Scapy:

from scapy.all import *
pkt = IP(dst="10.0.0.1")/TCP(dport=443)
send(pkt, iface="eth0", verbose=1)

Combine with iptables logging to verify marking:

iptables -t mangle -A PREROUTING -j LOG --log-prefix "FWDEBUG: "
iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark 0x100

This Bash script automates route testing for multiple scenarios:

#!/bin/bash

test_route() {
    local dest=$1
    local mark=$2
    echo "Testing route to $dest with fwmark $mark"
    ip route get $dest fwmark $mark
}

# Test common scenarios
test_route 8.8.8.8 0x100
test_route 192.168.100.1 0x200
test_route 10.10.10.10 0x300

When debugging intricate routing policies:

1. First verify base connectivity without marks
2. Check rule priority ordering: ip rule list
3. Verify table contents: ip route show table <id>
4. Use iptables -t mangle -L -v -n to confirm marking