Automating systemd Daemon Reload in SaltStack When Service Files Are Modified


2 views

When managing systemd services through SaltStack, a common pain point emerges: after deploying modified service files (.service, .timer, or other unit files), systemd requires a daemon reload to recognize the changes. The manual alternative looks like this:

# After deploying /etc/systemd/system/nginx.service
systemctl daemon-reload
systemctl restart nginx

This becomes tedious in automated environments and can lead to inconsistencies if forgotten.

SaltStack provides two elegant approaches to handle this automatically:

1. Using the watch Directive

In your SLS file, make the service watch the configuration file:

nginx_service:
  file.managed:
    - name: /etc/systemd/system/nginx.service
    - source: salt://nginx/files/nginx.service
    - template: jinja
  
nginx_reload:
  service.running:
    - name: nginx
    - enable: True
    - watch:
      - file: nginx_service
    - reload: True

The watch directive ensures the service restarts when the file changes, but doesn't handle the daemon-reload.

2. The Complete Solution with module.run

For full automation including daemon-reload:

deploy_nginx_service:
  file.managed:
    - name: /etc/systemd/system/nginx.service
    - source: salt://nginx/files/nginx.service
    - template: jinja

reload_systemd:
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: deploy_nginx_service

restart_nginx:
  service.running:
    - name: nginx
    - enable: True
    - watch:
      - file: deploy_nginx_service

For environments with numerous services, consider this DRY approach:

{% set services = ['nginx', 'postgresql', 'redis'] %}

{% for service in services %}
deploy_{{ service }}_service:
  file.managed:
    - name: /etc/systemd/system/{{ service }}.service
    - source: salt://{{ service }}/files/{{ service }}.service

reload_systemd_for_{{ service }}:
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: deploy_{{ service }}_service

restart_{{ service }}:
  service.running:
    - name: {{ service }}
    - watch:
      - file: deploy_{{ service }}_service
{% endfor %}

After applying these states, verify with:

salt 'minion-id' state.apply your_service_state test=True

Then check the systemd journal for reload events:

journalctl -u systemd --since "1 hour ago" | grep reload

For more control, you can check file hashes before reloading:

check_service_hash:
  file.hash:
    - name: /etc/systemd/system/your.service
    - algorithm: sha256
    - unless: test -z "$(systemctl show your.service -p LoadState --value | grep -q 'not-found')"

conditional_reload:
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: check_service_hash

When managing services through SaltStack, a common pain point emerges when modifying systemd unit files. After deploying changes to files like /etc/systemd/system/superfoo.service, systemd requires a daemon-reload to recognize the updates. Without this step, you'll see warnings like:

Warning: Unit file of superfoo.service changed on disk,
         'systemctl --system daemon-reload' recommended.

The most straightforward approach uses Salt's module.run with the service.systemctl_reload function. Here's a state example:

reload_systemd:
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: /etc/systemd/system/superfoo.service

For more complex scenarios, we can create a custom Salt execution module:

# _modules/custom_systemd.py
import os

def daemon_reload_if_changed(service_name):
    """
    Reload systemd if service file was modified
    """
    service_path = f"/etc/systemd/system/{service_name}"
    if os.path.exists(service_path):
        __salt__['service.systemctl_reload']()
        return True
    return False

Then in your state file:

include:
  - custom_systemd

reload_superfoo:
  module.run:
    - name: custom_systemd.daemon_reload_if_changed
    - m_name: superfoo
    - onchanges:
      - file: /etc/systemd/system/superfoo.service

Here's a full implementation that handles both the service file deployment and automatic reload:

deploy_superfoo_service:
  file.managed:
    - name: /etc/systemd/system/superfoo.service
    - source: salt://services/superfoo.service
    - mode: 644
    - user: root
    - group: root

reload_systemd_daemon:
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: deploy_superfoo_service

ensure_superfoo_running:
  service.running:
    - name: superfoo
    - enable: True
    - reload: True
    - require:
      - file: deploy_superfoo_service
      - module: reload_systemd_daemon

For environments with many services, we can use Jinja templating:

{% set services = ['service1', 'service2', 'service3'] %}

{% for svc in services %}
deploy_{{ svc }}_service:
  file.managed:
    - name: /etc/systemd/system/{{ svc }}.service
    - source: salt://services/{{ svc }}.service
    - mode: 644

{% endfor %}

systemd_daemon_reload:
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      {% for svc in services %}
      - file: deploy_{{ svc }}_service
      {% endfor %}