An open DNS resolver is a publicly accessible DNS server that responds to recursive queries from any IP address. When misconfigured, attackers can abuse it for DNS amplification attacks - a type of DDoS where small queries generate massive responses.

// Typical DNS amplification attack flow:
1. Attacker spoofs victim's IP address
2. Sends small DNS queries (60 bytes) to your open resolver
3. Your server responds with large replies (3000+ bytes) to victim
4. Traffic multiplies 50-100x in volume

Since your server runs Active Directory DNS, these registry tweaks are critical:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DNS\Parameters]
"EnableDirectoryPartitions"=dword:00000000
"DisableAutoReverseZones"=dword:00000001
"SocketPoolSize"=dword:00000000

In your DNS Manager console:

1. Disable recursion for external zones
2. Configure query filtering
3. Restrict zone transfers
4. Enable Response Rate Limiting (RRL)

Create these inbound rules in Windows Firewall:

netsh advfirewall firewall add rule name="Block External DNS" dir=in protocol=udp localport=53 action=block remoteip=!192.168.0.0/16,!10.0.0.0/8,!172.16.0.0/12
netsh advfirewall firewall add rule name="Allow Internal DNS" dir=in protocol=udp localport=53 action=allow remoteip=192.168.0.0/16,10.0.0.0/8,172.16.0.0/12

Use this PowerShell script to detect suspicious traffic:

Get-NetUDPSetting -LocalPort 53 | 
Where-Object { $_.ReceivedCount -gt 1000 } | 
ForEach-Object {
    Write-Warning "Potential DNS attack from $($_.RemoteAddress)"
    $_.RemoteAddress | Out-File "dns_attackers.txt" -Append
}

Consider separating your DNS roles:

• Internal authoritative-only DNS for AD
• External recursive resolver with proper ACLs
• Cloud-based DNS protection services

When your DNS server responds to recursive queries from any IP address (not just your internal network), it becomes an open resolver - essentially a public DNS server. Hackers exploit this by sending spoofed UDP requests where the source IP is their actual target. Your server then amplifies the attack by sending large DNS responses to the victim.

A typical attack chain looks like:

1. Attacker sends small DNS query (60 bytes) to your open resolver
2. Your server responds with large DNS record (3000+ bytes) 
3. Response is sent to victim's IP (spoofed source)
4. Multiply this by thousands of queries/second → DDoS

Windows Server DNS role has these default behaviors that create risk:

• Recursion enabled for all interfaces
• No source IP restrictions for recursive queries
• EDNS (Extension Mechanisms for DNS) allowing large payloads

Here's the PowerShell script I use to lock down DNS servers in AD environments:

# Disable recursion on public interfaces
Set-DnsServerRecursion -Enable $false -Force

# Configure recursion only for internal subnets
Add-DnsServerRecursionScope -Name "Internal" -EnableRecursion $true
Set-DnsServerRecursionScope -Name "Internal" -Subnet "192.168.0.0/16,10.0.0.0/8"

# Disable EDNS to prevent amplification
Set-DnsServerEDns -Enable $false

# Configure RRL (Response Rate Limiting)
Add-DnsServerResponseRateLimiting -Enabled $true -Mode "LogOnly"
Set-DnsServerResponseRateLimitingExceptionlist -Add "192.168.*.*"

# Enable DNS logging for monitoring
Set-DnsServerDiagnostics -All $true -EnableLogFileRollover $true

Even with DNS service hardening, implement these Windows Firewall rules:

# Block external DNS queries on UDP 53 (except from internal IPs)
New-NetFirewallRule -DisplayName "DNS External Block" -Direction Inbound -Protocol UDP -LocalPort 53 -RemoteAddress "!192.168.0.0/16,!10.0.0.0/8" -Action Block

# Rate limit DNS queries to prevent flooding
New-NetFirewallRule -DisplayName "DNS Rate Limit" -Direction Inbound -Protocol UDP -LocalPort 53 -Action RateLimit -RateLimitIntervalSec 60 -RateLimitBytes 10240

Use these tools to test your resolver:

# Using dig to test recursion (external perspective)
dig +short test.openresolver.com TXT @your.server.ip

# Using nmap for comprehensive scan
nmap -sU -p 53 --script dns-recursion your.server.ip

A secure configuration should return no response or a "recursion not allowed" message to external queries.