When running salt '*' test.ping
on a SaltStack master, the command often hangs while waiting for unresponsive minions to reply. This creates unnecessary delays when you simply want to see which minions are currently connected to your master.
Use the salt-key
command with the -L
flag to list all minions that have established connection with the master:
salt-key -L --out=json
This will return output in three categories:
{
"minions": ["minion1", "minion2"],
"minions_pre": [],
"minions_rejected": []
}
The minions
list shows all currently accepted and connected minions.
For more detailed connection information, you can use:
1. Event System Query:
salt-run state.event pretty=True | grep minion_connected
2. Master Job Cache:
salt-run jobs.list_jobs | grep -E 'Function: test.ping'
For regular monitoring, create a simple script:
#!/bin/bash
# Get connected minions and save to file
salt-key -L --out=json > /var/log/salt/minion_status.json
# Optional: Send alert for disconnected minions
grep -q \"minions\":\$$\$$ /var/log/salt/minion_status.json && \
echo "Alert: No minions connected!" | mail -s "SaltStack Alert" admin@example.com
For programmatic access, use Salt's REST API:
curl -sS localhost:8000/keys | jq '.return.minions'
Remember to configure the REST API in your master config first:
rest_cherrypy:
port: 8000
disable_ssl: true
When working with SaltStack, the salt '*' test.ping
command is commonly used to check minion availability. However, this approach has a significant limitation - it only shows minions that are both connected and responsive. In production environments, you often need to see all minions that have active connections to the master, regardless of their current responsiveness.
Salt Master maintains connection state information in the following directory:
/var/cache/salt/master/minions/
Each connected minion has a corresponding directory here containing its connection state files. This persists even if the minion becomes temporarily unresponsive.
Here are several reliable methods to list all connected minions:
Using salt-key
salt-key --list all
This shows all minions that have keys accepted by the master, including those not currently connected.
Checking the Minions Cache Directory
ls /var/cache/salt/master/minions/
This directly lists all minions that have active connections to the master.
Using the Salt Runner
salt-run manage.up
While similar to test.ping
, this runner provides better performance for large deployments.
For more control, you can use Salt's Python API:
import salt.client
client = salt.client.LocalClient()
connected = client.cmd('*', 'test.ping', timeout=2, full_return=True)
# Filter responsive minions
responsive = [minion for minion, data in connected.items() if data['ret']]
print(f"Responsive minions: {responsive}")
When dealing with thousands of minions, consider these optimizations:
- Use
--async
flag for non-blocking operations - Increase timeout values for large deployments
- Consider using job cache for historical data
If minions appear disconnected but should be online:
salt-run manage.status
salt-run jobs.active
These commands help identify if the master is overloaded or if there are network issues.