When working with GitLab, SSH key authentication issues can be particularly frustrating because they prevent basic Git operations like clone, pull, and push. The symptoms you're seeing - 403 errors with HTTP and "permission denied" with SSH - typically indicate authentication problems.
First, let's verify some critical components:
# Check if SSH daemon is running
sudo systemctl status sshd
# Verify GitLab's SSH configuration
cat /etc/gitlab/gitlab.rb | grep ssh
The SSH config file (~/.ssh/config) you've created looks correct, but there might be permission issues:
# Verify file permissions
ls -la ~/.ssh/
# Correct permissions if needed
chmod 600 ~/.ssh/config
chmod 600 ~/.ssh/id_rsa
The /var/log/secure entries showing "Connection closed" suggest the SSH handshake is failing. Let's enable more verbose logging:
# Edit SSH daemon config
sudo vim /etc/ssh/sshd_config
# Add or modify these lines:
LogLevel VERBOSE
PubkeyAuthentication yes
Before trying Git operations, test the SSH connection directly:
ssh -Tv git@your-gitlab-server.com
This verbose output (-v) will show exactly where the authentication fails.
For GitLab installations, there are some special considerations:
# Reconfigure GitLab after changes
sudo gitlab-ctl reconfigure
# Check GitLab's authorized_keys file
sudo cat /var/opt/gitlab/.ssh/authorized_keys
If you're using Windows with Git Bash, try:
# Start SSH agent
eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa
If SSH continues to fail, consider temporary alternatives:
# Use HTTPS with credential caching
git config --global credential.helper cache
git clone https://your-gitlab-server.com/your-repo.git
After making changes, verify everything works:
# Test Git operations
git clone git@your-gitlab-server.com:your-repo.git
cd your-repo
touch test.txt
git add test.txt
git commit -m "Test commit"
git push origin master
The problem manifests when trying to perform Git operations (clone/pull/push) against a GitLab instance where:
- Web UI functions normally
- SSH connections get terminated abruptly (Connection closed by [IP])
- HTTP attempts result in 403 errors
- Public keys appear properly registered in GitLab
First let's verify the SSH client configuration. The provided ~/.ssh/config
appears correct, but we should add debug flags:
Host gitlab.example.com
User git
Hostname gitlab.example.com
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_rsa
LogLevel DEBUG3
The /var/log/secure
logs show connection closures without authentication attempts. This suggests either:
- SSH daemon configuration issues
- Filesystem permission problems
- GitLab's authorized_keys handling
Check the GitLab Shell logs:
sudo tail -f /var/log/gitlab/gitlab-shell/gitlab-shell.log
Run these diagnostic commands on the GitLab server:
# Verify git user's home directory permissions
sudo ls -ld /var/opt/gitlab/
sudo ls -la /var/opt/gitlab/.ssh/
# Check GitLab's authorized_keys file
sudo cat /var/opt/gitlab/.ssh/authorized_keys
# Validate SELinux context if applicable
sudo ls -Z /var/opt/gitlab/.ssh/authorized_keys
Common solutions include:
# Regenerate authorized_keys
sudo gitlab-rake gitlab:shell:setup
# Reconfigure GitLab
sudo gitlab-ctl reconfigure
# Verify SSH daemon configuration
sudo grep -i "AcceptEnv" /etc/ssh/sshd_config
Test SSH connectivity manually:
ssh -Tv git@gitlab.example.com
# If connection succeeds but Git operations fail:
GIT_SSH_COMMAND="ssh -v" git clone git@gitlab.example.com:user/repo.git
For Windows clients using Pageant:
- Ensure Pageant is running with your private key loaded
- Verify GIT_SSH environment variable points to plink.exe
- Check for line ending issues in keys:
# In git-bash:
dos2unix ~/.ssh/id_rsa
If SELinux is enforcing:
# Check for denials
sudo ausearch -m avc -ts recent
# Temporary disable to test
sudo setenforce 0
# If problem resolves:
sudo semanage fcontext -a -t ssh_home_t "/var/opt/gitlab/.ssh/authorized_keys"
sudo restorecon -v /var/opt/gitlab/.ssh/authorized_keys