How to Run Multiple Instances of a Parameterless Script Using GNU Parallel


2 views

When trying to execute multiple instances of a script that requires no arguments using GNU Parallel, you might encounter the warning:

parallel: Warning: Input is read from the terminal. Only experts do this on purpose. Press CTRL-D to exit.

This happens because GNU Parallel expects input by default, even when your script doesn't need any parameters.

The simplest way to run N instances of your script:

parallel -N0 foo.sh ::: {1..N}

Where N is the number of instances you want to run. The -N0 tells Parallel to use zero arguments per job.

You have several other options depending on your specific needs:

# Using a dummy input source
yes | parallel -N0 foo.sh | head -n N

# Using seq
seq N | parallel -N0 foo.sh

# Using a null input file
touch emptyfile
parallel -a emptyfile foo.sh

For more control over parallel execution:

# Run 8 instances with progress reporting
parallel -j8 --progress -N0 foo.sh ::: {1..100}

# Limit CPU usage
parallel --load 80% -N0 foo.sh ::: {1..100}

# Distribute across servers
parallel -S server1,server2 -N0 foo.sh ::: {1..100}

Be aware of these potential issues:

  • Scripts that modify global state may cause race conditions
  • Output from parallel jobs may interleave unpredictably
  • Resource contention may occur if not properly managed

When running many instances:

# Monitor system resources
parallel --bar -N0 foo.sh ::: {1..100} | pv -l >/dev/null

# Adjust batch size for optimal performance
parallel --block-size 10M -N0 foo.sh ::: {1..1000}

When running scripts that don't accept arguments through GNU Parallel, you'll encounter a warning about reading from terminal input. This happens because Parallel expects to distribute input arguments by default.

The most straightforward way to handle this is to provide a fixed number of dummy inputs:

parallel foo.sh ::: {1..10}

This runs foo.sh 10 times (with dummy arguments 1 through 10 that the script will ignore).

If you need to run the script exactly N times without generating dummy arguments:

seq N | parallel foo.sh

Or using Parallel's built-in features:

parallel --halt now,fail=1 foo.sh ::: dummy_input

For a CPU-intensive simulation script called simulate.sh that needs 20 parallel runs:

parallel -j 20 simulate.sh ::: {1..20}

You can combine this with other Parallel features for better control:

parallel --progress --joblog simulation.log simulate.sh ::: {1..20}

Remember that:

  • Each instance still consumes system resources
  • Too many parallel runs may cause swapping or OOM kills
  • Use -j to limit concurrent jobs if needed