When implementing HTTPS redirection while using Let's Encrypt's webroot authentication method, we encounter a catch-22 situation. The ACME validation requires access to the /.well-known/acme-challenge/ directory via HTTP (port 80), but our standard practice is to redirect all HTTP traffic to HTTPS.

Here's the proper nginx configuration that allows HTTP access for Let's Encrypt validation while redirecting all other traffic:

server {
    listen 80;
    server_name sub.domain.tld;
    server_tokens off;

    # Let's Encrypt challenge directory
    location ^~ /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
        try_files $uri =404;
    }

    # Redirect everything else to HTTPS
    location / {
        return 301 https://$host$request_uri;
    }
}

The initial attempt used a simple location /.well-known directive, which doesn't have the same precedence as the ^~ modifier. The caret-tilde prefix makes nginx:

1. Stop searching for more specific matches
2. Give this location block higher priority than regex matches

After implementing this configuration, verify it works correctly:

# Test configuration syntax
sudo nginx -t

# Reload nginx
sudo systemctl reload nginx

# Verify HTTP access to challenges
curl -I http://sub.domain.tld/.well-known/acme-challenge/test

# Verify HTTPS redirection
curl -I http://sub.domain.tld/some-page

Here's a full server block implementation including the HTTPS portion:

server {
    listen 80;
    server_name sub.domain.tld;
    
    location ^~ /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
        try_files $uri =404;
    }
    
    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name sub.domain.tld;
    
    ssl_certificate /etc/letsencrypt/live/sub.domain.tld/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/sub.domain.tld/privkey.pem;
    
    # Other SSL configurations...
    
    location / {
        # Your regular content handling
        root /var/www/html;
        try_files $uri $uri/ =404;
    }
}

When implementing this pattern:

• Ensure the /var/www/letsencrypt directory has proper permissions (typically owned by www-data or nginx user)
• Consider adding rate limiting to the challenge directory to prevent brute force attacks
• Log access to the challenge directory separately for monitoring

When setting up HTTPS with Let's Encrypt on Nginx, many developers face a common issue: the ACME challenge requires access to /.well-known/acme-challenge/ via HTTP (port 80), but their server configuration redirects all HTTP traffic to HTTPS. This creates a chicken-and-egg problem where the certificate can't be obtained because the verification requests are being redirected.

The typical Nginx configuration that causes this issue looks like:

server {
    listen 80;
    server_name example.com;
    return 301 https://$host$request_uri;
}

This configuration unconditionally redirects all HTTP requests to HTTPS, including the Let's Encrypt validation requests.

We need to modify our Nginx configuration to exclude the /.well-known directory from the HTTP to HTTPS redirect. Here's the correct approach:

server {
    listen 80;
    server_name example.com;
    
    location /.well-known {
        root /var/www/letsencrypt;
        try_files $uri =404;
    }
    
    location / {
        return 301 https://$host$request_uri;
    }
}

Let's break down why this works:

• The location /.well-known block matches requests to the ACME challenge directory
• The root directive specifies where the challenge files are stored
• The try_files directive ensures Nginx looks for the exact file requested
• All other locations fall through to the redirect

If you're still having issues, check these common problems:

# Incorrect: Missing the leading slash
location .well-known { ... }

# Incorrect: Using alias instead of root
location /.well-known {
    alias /var/www/letsencrypt/.well-known;
}

# Correct: Using root with full path
location /.well-known {
    root /var/www/letsencrypt;
}

Before running certbot, test your configuration with:

sudo nginx -t
sudo systemctl reload nginx

Then verify the challenge is accessible:

curl http://example.com/.well-known/acme-challenge/testfile

Here's a full server block that works with Let's Encrypt:

server {
    listen 80;
    server_name example.com www.example.com;
    
    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
        try_files $uri =404;
    }
    
    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name example.com www.example.com;
    
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    
    # ... rest of your HTTPS configuration ...
}

Remember to create the directory and set proper permissions:

sudo mkdir -p /var/www/letsencrypt/.well-known/acme-challenge
sudo chown -R www-data:www-data /var/www/letsencrypt
sudo chmod -R 755 /var/www/letsencrypt