When administering Slurm clusters, a common frustration arises when trying to modify user-account associations. The intuitive command:
sacctmgr modify user where name=example set account=groupb
often fails with errors because Slurm's design requires users to be explicitly associated with accounts through a separate relationship table.
Slurm treats user-account relationships as separate entities from user records themselves. This design allows for:
- Multiple account associations per user
- Different QOS settings per association
- Flexible permission management
Here are three working approaches to change account associations:
Method 1: Delete and Recreate (Basic)
sacctmgr delete user name=example account=groupa
sacctmgr add user name=example account=groupb
Method 2: Direct Association Modification (Recommended)
sacctmgr modify user name=example set defaultaccount=groupb
sacctmgr modify association where user=example and account=groupa set account=groupb
Method 3: Batch Modification (For Multiple Users)
sacctmgr modify user set defaultaccount=groupb where account=groupa
sacctmgr modify association set account=groupb where account=groupa
Always verify changes with:
sacctmgr show user example
sacctmgr show association user=example
Error: "Association not found"
Solution: First ensure the target account exists:
sacctmgr show account groupb
Error: "Permission denied"
Solution: Run as Slurm administrator or use sudo:
sudo sacctmgr [command]
Warning: Pending jobs
Solution: Either wait for jobs to complete or force requeue:
scontrol requeue $(squeue -u example -h -o "%i")
For production environments, use transaction mode to prevent partial updates:
sacctmgr -i modify user name=example set defaultaccount=groupb
sacctmgr -i modify association where user=example set account=groupb
Many administrators face frustration when attempting to modify user associations in Slurm using the intuitive sacctmgr modify
command. The typical approach:
sacctmgr modify user where name=example set account=groupb
Often fails with errors because Slurm requires additional context for user-account relationships. This isn't immediately obvious from the documentation.
Slurm treats user-account relationships as first-class entities called "associations." When you run:
sacctmgr show associations
You'll see output like:
Cluster Account User Partition Share GrpJobs GrpTRES GrpSubmit
------- ------- ---- --------- ----- ------- ------- ---------
cluster1 groupa example batch 1 0 cpu=0 0
cluster1 groupb - batch 1 0 cpu=0 0
Notice that user associations are distinct from account definitions. The hyphen (-) under User indicates a pure account definition.
To properly move a user between groups, you must specify both the source and target associations:
sacctmgr modify user where name=example and account=groupa set account=groupb
Key points about this command:
- Explicitly identifies the current association (name=example AND account=groupa)
- Modifies only that specific association
- Preserves all other user attributes
After modification, verify with:
sacctmgr show user example
Should show output like:
User Def Acct Admin
---- -------- -----
example groupb None
And confirm with:
sacctmgr show associations user=example
For cases where users have multiple associations (rare but possible), you might need:
sacctmgr modify user where name=example set defaultaccount=groupb
sacctmgr modify association where user=example and account=groupa set account=groupb
This two-step approach ensures both the default account and specific associations are updated.
If you receive "Nothing modified" errors:
- Verify both source and target accounts exist
- Check for typos in user/account names
- Ensure you have sufficient privileges
- Try the verbose flag:
sacctmgr -v modify ...
Remember that Slurm associations are hierarchical. Changes may propagate to child accounts depending on your configuration.