How to Make dig +short Return Consistent Output for Empty DNS Responses in Batch Processing


2 views

When working with DNS queries in bulk processing scenarios, the default behavior of dig +short creates significant headaches. Consider this common situation:

dig +short -f queries.txt
# queries.txt contents:
A example.com
TXT nonexistent-record.com
A google.com

The output might look like:

93.184.216.34
216.58.214.46

This becomes problematic when:

  • Matching query input with output lines
  • Processing results in scripts
  • Generating reports
  • Piping to other commands like paste

Here are several approaches to maintain consistent output formatting:

1. Using bash Script Wrapper

#!/bin/bash
while read -r query; do
  result=$(dig +short $query)
  [ -z "$result" ] && echo "NOANSWER" || echo "$result"
done < queries.txt

2. Leveraging dig's +noall +answer

dig +noall +answer -f queries.txt | awk '{print $NF}'

3. Python Alternative

import dns.resolver

with open('queries.txt') as f:
    for line in f:
        qtype, domain = line.strip().split()
        try:
            answers = dns.resolver.resolve(domain, qtype)
            for rdata in answers:
                print(rdata.to_text())
        except:
            print("NOANSWER")

Create a wrapper function in your shell profile:

dig_consistent() {
  while IFS= read -r line; do
    output=$(dig +short $line)
    printf "%-50s %s\n" "$line" "${output:-NOANSWER}"
  done < "${1:-/dev/stdin}"
}

For mixed query types in your input file, this enhanced version preserves the original query:

dig +noall +answer -f queries.txt | \
  awk '{if (NF < 5) print $1 " " $2 " NOANSWER"; else print $1 " " $2 " " $NF}'

When working with DNS queries in bulk processing, the default behavior of dig +short creates a significant parsing challenge. Consider this common scenario:

dig +short -f queries.txt
# queries.txt contents:
A example.com
TXT non-existent-record.com
A example.org

The output might look like this:

93.184.216.34
93.184.216.34

Notice how the empty response for the TXT query completely disappears from the output stream, breaking the correspondence between input lines and output results.

Here are several approaches to maintain output alignment:

1. Using +noall +answer with Custom Formatting

dig +noall +answer -f queries.txt | awk '{print $NF}'
# Alternative:
dig +noall +answer -f queries.txt | sed -E 's/.*[\t ]//'

This preserves empty lines but still doesn't guarantee 1:1 mapping.

2. Shell Wrapper Script

Create a wrapper script (e.g., dig_wrapper.sh):

#!/bin/bash
while read -r query; do
  result=$(dig +short $query)
  [ -z "$result" ] && echo "NO_ANSWER" || echo "$result"
done

Usage:

cat queries.txt | ./dig_wrapper.sh

3. Advanced awk Processing

Combine dig with awk for robust processing:

dig -f queries.txt +noall +answer | awk '
BEGIN { RS=""; FS="\n"; OFS="\n" }
{
  split($1, q, "[\t ]");
  if (NF > 1) {
    for (i=2; i<=NF; i++) {
      split($i, a, "[\t ]");
      print a[NF]
    }
  } else {
    print "NO_ANSWER"
  }
}'

For production systems, consider these alternatives:

# Using kdig (from Knot DNS)
kdig +short +queries=queries.txt

# Using dog (Rust alternative)
dog --short -N @1.1.1.1 -f queries.txt

For mission-critical batch processing, I recommend implementing a solution that:

  1. Preserves 1:1 input-output mapping
  2. Explicitly marks empty responses
  3. Includes query metadata in output

Here's a production-ready Python solution:

import subprocess
import sys

def batch_dig(query_file):
    with open(query_file) as f:
        queries = [line.strip() for line in f]
    
    results = []
    for query in queries:
        try:
            output = subprocess.check_output(
                ['dig', '+short'] + query.split(),
                stderr=subprocess.PIPE,
                text=True
            )
            results.append(output.strip() or "NO_ANSWER")
        except subprocess.CalledProcessError:
            results.append("QUERY_ERROR")
    
    return results

if __name__ == "__main__":
    for result in batch_dig(sys.argv[1]):
        print(result)