How to Stream Real-Time Shell Command Output in Ansible Playbooks


3 views

When running shell commands through Ansible playbooks, many developers encounter a frustrating limitation - the output only appears after the command completes, rather than streaming in real-time as it would in a terminal. This makes it difficult to monitor long-running processes or debug commands interactively.

- name: Example showing delayed output
  hosts: localhost
  tasks:
    - name: Run apt update
      shell: apt update
      register: apt_result

    - name: Show output
      debug:
        var: apt_result.stdout

Ansible processes command output through its internal execution engine, which buffers the output until the task completes. This design ensures clean logging and reliable execution tracking, but sacrifices real-time visibility.

Method 1: Using the Debug Callback Plugin

Enable Ansible's debug callback to stream output:

ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook playbook.yml

Method 2: Directly Piping to stdout

Force immediate output by piping through tee:

- name: Stream apt update output
  shell: "apt update | tee /dev/stderr"
  args:
    executable: /bin/bash

Method 3: Custom Callback Plugin

Create a custom callback plugin (save as callback_plugins/stream_output.py):

from ansible.plugins.callback import CallbackBase

class CallbackModule(CallbackBase):
    def v2_runner_on_ok(self, result):
        if 'cmd' in result._task.action:
            print(result._result.get('stdout', ''))

Then run with:

ANSIBLE_CALLBACK_PLUGINS=./callback_plugins ansible-playbook playbook.yml
  • Real-time output may interfere with Ansible's output formatting
  • Some commands buffer their own output (use stdbuf for these cases)
  • For production, consider logging to files instead
- name: Comprehensive output streaming example
  hosts: localhost
  gather_facts: no

  tasks:
    - name: Method 1 - Debug callback
      shell: echo "Streaming via debug callback"
      when: false  # Example only

    - name: Method 2 - Tee to stderr
      shell: "echo 'Streaming via tee' | tee /dev/stderr"

    - name: Method 3 - Custom callback
      shell: echo "Streaming via custom callback"
      register: callback_out
      when: false  # Requires callback plugin

For years, Ansible users have struggled with the default behavior of shell command output buffering. When executing commands through the shell or command modules, Ansible collects all output and only displays it after the command completes. This makes it difficult to:

  • Monitor long-running processes
  • Debug hanging operations
  • See progress indicators

The standard output handling in Ansible was designed for idempotency and clean logging, but it creates problems for several common scenarios:

- name: Run package update
  shell: apt-get update
  # Only shows 'changed' status, no output until complete

- name: Compile large application
  shell: make -j8
  # No visibility into compilation progress

- name: Download large file
  shell: wget https://example.com/large.iso
  # No download progress visible

While the GitHub issue (#3887) suggests this isn't possible directly, callback plugins provide a workaround. Here's how to implement real-time output:

# Save this as callback_plugins/realtime.py
from ansible.plugins.callback import CallbackBase
import sys

class CallbackModule(CallbackBase):
    def v2_runner_on_ok(self, result):
        if 'cmd' in result._task.action:
            sys.stdout.write(result._result.get('stdout', ''))
            sys.stdout.flush()

Enable the callback plugin in your ansible.cfg:

[defaults]
stdout_callback = realtime
callback_whitelist = realtime

Now run your playbook with increased verbosity:

ansible-playbook playbook.yml -v

For more control, consider these approaches:

  1. Line buffering with stdbuf:
    - name: Run command with line buffering
      shell: stdbuf -oL apt-get update
    
  2. Using the raw module:
    - name: Bypass module buffering
      raw: apt-get update
    
  3. Custom script redirection:
    - name: Stream output to file
      shell: "apt-get update | tee /var/log/update.log"
      async: 3600
      poll: 0
    - name: Follow log
      command: tail -f /var/log/update.log
      register: tail_output
      until: tail_output.stdout.find("Reading package lists") != -1
      retries: 30
      delay: 10