How to Handle Googlebot IP Masquerading in User Registration: Technical Solutions for IP-based Geo-Restrictions


1 views

When implementing IP-based geo-restrictions on your website, you might encounter users appearing to connect from Google's IP ranges (like 66.249.93.202). This occurs primarily due to:

  • Google's Accelerated Mobile Pages (AMP) cache serving your content
  • Google's proxy services for various products
  • Mobile carriers routing traffic through Google's infrastructure

The core problem stems from how modern mobile networks and Google services handle traffic:

// Example of problematic IP detection logic
function isEuropeanUser(ip) {
    const europeanRanges = ['193.0.0.0/8', '194.0.0.0/7'];
    return europeanRanges.some(range => isIPInRange(ip, range));
}

This simplistic approach fails when traffic comes through Google's infrastructure.

1. Client-Side IP Verification

Implement a secondary verification using JavaScript:

// Using a third-party IP detection service
async function getRealIP() {
    try {
        const response = await fetch('https://api.ipify.org?format=json');
        const data = await response.json();
        return data.ip;
    } catch (error) {
        return null;
    }
}

2. Server-Side X-Forwarded-For Header Check

Modify your server logic to check headers:

// PHP example for checking X-Forwarded-For
function getClientIP() {
    if (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
        $ips = explode(',', $_SERVER['HTTP_X_FORWARDED_FOR']);
        return trim($ips[0]);
    }
    return $_SERVER['REMOTE_ADDR'];
}

3. Combining Multiple Detection Methods

A robust solution should combine several techniques:

// Node.js example combining methods
const requestIp = require('request-ip');
const geoip = require('geoip-lite');

app.use((req, res, next) => {
    const clientIp = requestIp.getClientIp(req) || req.ip;
    const geo = geoip.lookup(clientIp);
    
    if (geo && geo.country !== 'ES') {
        return res.status(403).send('Access denied');
    }
    next();
});

For mobile users, consider these additional approaches:

  • Implement CAPTCHA challenges for suspected proxy traffic
  • Use WebRTC to detect local IP addresses (with user permission)
  • Check for Googlebot-specific headers when appropriate
// Detecting Googlebot traffic
function isGooglebot(req) {
    const userAgent = req.headers['user-agent'] || '';
    const viaHeader = req.headers['via'] || '';
    return userAgent.includes('Googlebot') || viaHeader.includes('google');
}
Approach Implementation Level Accuracy
X-Forwarded-For check Server Medium
Third-party IP API Client/Server High
WebRTC detection Client High
User-Agent analysis Server Low

When implementing geo-based authentication restrictions, developers often encounter cases where legitimate users appear to be connecting from Google's IP ranges (like 66.249.93.202). This occurs because:

  • Google's Accelerated Mobile Pages (AMP) cache serves your content
  • Chrome Data Saver/Lite mode proxies traffic
  • Android devices using Google's fetch service
  • Some mobile carriers route through Google proxies

Google services act as intermediaries in these scenarios:

Original flow: [User Device] → [Your Server]
Proxied flow: [User Device] → [Google Proxy] → [Your Server]

The X-Forwarded-For header becomes crucial for identifying real client IPs:

// PHP example to check forwarded IP
$forwarded_ips = explode(',', $_SERVER['HTTP_X_FORWARDED_FOR']);
$real_ip = trim($forwarded_ips[0]) ?: $_SERVER['REMOTE_ADDR'];

Method 1: Header Inspection

// Node.js middleware example
app.use((req, res, next) => {
  const isGoogleIP = require('google-ip-ranges').contains(req.ip);
  const realIP = req.headers['x-forwarded-for']?.split(',')[0] || req.ip;
  
  if (isGoogleIP && !isEUCountry(realIP)) {
    return res.status(403).json({ error: 'Please disable Data Saver mode' });
  }
  next();
});

Method 2: Client-Side Detection

// JavaScript detection for AMP
if (window.location.href.includes('google.com/amp/')) {
  document.cookie = 'amp_user=true; path=/';
}

// Backend check
if ($_COOKIE['amp_user'] && validateForwardedIP()) {
  // Special handling for AMP users
}

For Android Chrome Data Saver users, implement these workarounds:

  • Detect the Save-Data: on header
  • Show a notification suggesting to disable Data Saver temporarily
  • Implement a fallback authentication method using device fingerprints

Always use multiple verification methods:

# Python example with maxminddb
import maxminddb

def validate_country(ip):
    with maxminddb.open_database('GeoLite2-Country.mmdb') as reader:
        country = reader.get(ip).get('country', {}).get('iso_code')
    
    # Secondary check for cloud providers
    if ip in google_ips and 'x-forwarded-for' in request.headers:
        return validate_country(request.headers['x-forwarded-for'].split(',')[0])
    
    return country

Remember to:

  • Regularly update your GeoIP databases
  • Monitor Google's published IP ranges
  • Implement graceful degradation for proxy users