When authentication issues arise in Linux systems, PAM (Pluggable Authentication Modules) often becomes the prime suspect. Yet debugging PAM can feel like chasing ghosts due to its silent failures. Here's how to force PAM to reveal its secrets through syslog.

For Debian-based systems (including Squeeze), these are the most effective approaches:

# Method 1: Global debug flag (works for most PAM modules)
sudo sed -i 's/^#@include common-auth/@include common-auth\nauth       optional     pam_exec.so debug log=/var/log/pam_debug.log/' /etc/pam.d/sshd

# Method 2: Module-specific debugging
sudo sed -i 's/pam_unix.so/pam_unix.so debug/' /etc/pam.d/common-auth

To ensure PAM logs appear in syslog:

# Create PAM-specific syslog rule
echo "auth.*;authpriv.*   /var/log/pam.log" | sudo tee -a /etc/rsyslog.conf

# Restart services
sudo systemctl restart rsyslog
sudo systemctl restart sshd  # or whatever service uses PAM

When basic logging isn't sufficient:

# Compile PAM with debug symbols (for Debian)
sudo apt-get install libpam0g-dev
cd /usr/src
sudo apt-get source pam
cd pam-*
./configure --enable-debug
make
sudo make install

For nullok_secure issues mentioned in the original post:

# Sample debug output when nullok_secure fails
PAM_DEBUG: [pam_unix.c:unix_verify_password:456] username=[testuser]
PAM_DEBUG: [pam_unix.c:unix_verify_password:482] nullok_secure check: tty=/dev/pts/1 secure=0
PAM_DEBUG: [pam_unix.c:unix_verify_password:495] authentication failed (nullok_secure restriction)

• Always test changes in a new terminal session - don't lock yourself out
• Use pam_tally2 for failed attempt debugging
• Check /var/log/faillog for complementary information
• For SSH issues, combine with sshd -ddd debugging

If you've ever struggled with mysterious PAM authentication issues in Debian (or any Linux distro), you know the frustration of trying to debug silent failures. The system either works or doesn't, with zero visibility into what's happening behind the scenes.

Here's how to force PAM to reveal its secrets through syslog:

# Edit your PAM service configuration (e.g., sshd)
sudo nano /etc/pam.d/sshd

# Add debug to modules you want to trace
auth       debug     pam_unix.so
session    debug     pam_limits.so

Ensure your syslog captures PAM debug messages (Debian uses rsyslog by default):

# Create a dedicated PAM logging file
sudo nano /etc/rsyslog.d/50-pam.conf

# Add these lines:
auth.*,authpriv.*    /var/log/pam.log
if $programname == 'pam_' then /var/log/pam-debug.log

When troubleshooting that pesky nullok_secure issue, debug output might show:

# Sample debug output
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4
pam_unix(sshd:auth): check pass; user unknown
pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4
pam_securetty(sshd:auth): access denied: tty 'ssh' is not secure!

For deeper troubleshooting, consider these approaches:

• Add debug to every PAM module temporarily
• Use pamtester to simulate authentication attempts
• Check /var/log/auth.log and your custom PAM logs simultaneously

You can enable additional debugging through environment variables:

# Set PAM debugging level
export PAM_DEBUG=1
export PAM_VERBOSE=1

# Then test with:
su - username

Always verify your PAM stack after changes:

# Check PAM configuration syntax
pam-auth-update --force

# Restart services after changes
sudo systemctl restart sshd rsyslog