When working with Apache Zookeeper in production environments, checking cluster status is a fundamental administrative task. The most straightforward method is using the stat
command via the Zookeeper CLI.
$ echo stat | nc localhost 2181
Zookeeper version: 3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT
Clients:
/192.168.1.100:44824[0](queued=0,recved=1,sent=0)
/192.168.1.101:34218[1](queued=0,recved=152,sent=152)
Latency min/avg/max: 0/0.5/23
Received: 153
Sent: 152
Connections: 2
Outstanding: 0
Zxid: 0x200000002
Mode: follower
Node count: 487
The Mode
field in the output shows whether the current node is a leader
or follower
. To find all cluster members and their roles:
$ echo mntr | nc localhost 2181
zk_version 3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a
zk_avg_latency 0
zk_max_latency 23
zk_min_latency 0
zk_packets_received 154
zk_packets_sent 153
zk_num_alive_connections 2
zk_outstanding_requests 0
zk_server_state follower
zk_znode_count 487
zk_watch_count 0
zk_ephemerals_count 0
zk_approximate_data_size 47895
zk_open_file_descriptor_count 32
zk_max_file_descriptor_count 1048576
For a complete view of the ensemble configuration:
$ echo conf | nc localhost 2181
clientPort=2181
secureClientPort=-1
dataDir=/var/lib/zookeeper/version-2
dataLogDir=/var/lib/zookeeper/version-2
tickTime=2000
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
serverId=2
initLimit=10
syncLimit=5
electionAlg=3
electionPort=3888
quorumPort=2888
peerType=0
For frequent monitoring, create a simple bash script:
#!/bin/bash
ZK_NODES=("zk1.example.com" "zk2.example.com" "zk3.example.com")
for node in "${ZK_NODES[@]}"; do
echo "Checking $node:"
echo -e "stat\nmntr" | nc $node 2181 | grep -E 'Mode|zk_server_state'
echo "------------------------"
done
This script will output each node's status in a readable format, making it easy to identify the leader at a glance.
Zookeeper supports several four-letter commands for diagnostics:
ruok
- Checks if server is running in non-error statesrvr
- Gives full server statscons
- Lists all active connectionscrst
- Resets connection stats counters
$ echo srvr | nc localhost 2181
Zookeeper version: 3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT
Latency min/avg/max: 0/0.5/23
Received: 157
Sent: 156
Connections: 2
Outstanding: 0
Zxid: 0x200000002
Mode: follower
Node count: 487
To inspect your ZooKeeper cluster's status, you'll need to use the zkCli.sh
or zkCli.cmd
utility that comes with ZooKeeper installation. First connect to any cluster node:
./zkCli.sh -server host:port
# Example:
./zkCli.sh -server zk1.example.com:2181
Once connected, these commands reveal cluster information:
# Get basic server stats
stat /
# Detailed server status
srvr
# List all cluster members (requires ZooKeeper 3.5+)
config | grep "^server\\."
# Get current leader information
stat / | grep -i leader
For automation or when zkCli isn't available:
# Using telnet/nc (replace 2181 with your client port)
echo stat | nc localhost 2181 | grep Mode
echo mntr | nc localhost 2181 | grep zk_server_state
# Four-letter commands via HTTP (if admin server enabled)
curl http://localhost:8080/commands/stat
A leader node will show:
Mode: leader
zk_server_state: leader
Followers will display:
Mode: follower
zk_server_state: follower
The mntr
command provides comprehensive metrics:
echo mntr | nc localhost 2181
# Key metrics to monitor:
zk_avg_latency
zk_outstanding_requests
zk_znode_count
zk_watch_count
zk_ephemerals_count
Here's a bash script to check all cluster members:
#!/bin/bash
ZK_NODES=("zk1:2181" "zk2:2181" "zk3:2181")
for node in "${ZK_NODES[@]}"; do
echo -n "$node: "
echo stat | nc ${node%:*} ${node#*:} | grep "Mode:"
done
Newer versions provide RESTful endpoints:
curl http://localhost:8080/commands/mntr
curl http://localhost:8080/cluster