How to Force xargs to Use Only Newlines as Separators in Bash (Linux/OSX)


2 views

When working with xargs in bash, the default behavior of using whitespace as separators often causes headaches when processing filenames or data containing spaces. While the -0 option for NULL-delimited input solves this, many common Unix tools don't support NULL byte output natively.

The most straightforward solution is to use xargs' -d option (available in GNU xargs and modern BSD variants):

grep -rl 'pattern' | xargs -d '\n' rm

This forces xargs to treat ONLY newlines as separators, while preserving all other whitespace in the input.

For systems where -d isn't available (older OSX versions), we can use these workarounds:

Using tr to convert newlines:

find . -name "*.txt" | tr '\n' '\0' | xargs -0 grep "search"

Using printf to append NULL bytes:

find . -print0 | xargs -0 ls -ld

Consider these common scenarios where newline-only separation is crucial:

Processing filenames with spaces:

find . -name "*.log" -print0 | xargs -0 -n 1 gzip

Multi-line data processing:

printf "file one.txt\nfile two.txt" | xargs -d '\n' -I{} mv {} {}.bak

When dealing with large datasets, the delimiter method can impact performance:

  • -d '\n' is generally faster than NULL-delimited methods
  • Combining with -P for parallel processing helps offset any overhead

When working with xargs in bash, the default behavior of treating whitespace (spaces, tabs, and newlines) as item separators often causes problems with filenames or data containing spaces. While -0 (NUL delimiter) is the proper solution, many standard Unix tools don't support NUL-delimited output.

To make xargs use only newlines as separators:

# Basic version
command_producing_output | xargs -d '\n' [your_command]

# Practical example with find
find . -type f -print | xargs -d '\n' ls -l

# Preserving all whitespace within items
printf "file with spaces.txt\nanother_file.log" | xargs -d '\n' -I{} echo "Processing: {}"

The -d option works consistently across:

  • GNU xargs (Linux default)
  • BSD xargs (macOS/OS X default)

When dealing with extremely old systems where -d might not be available:

# Using shell substitutions (slower for large inputs)
command_producing_output | while IFS= read -r line; do
    your_command "$line"
done

# Using process substitution
xargs -I{} your_command {} < <(command_producing_output | tr '\n' '\0') 

Common scenarios where newline-only separation matters:

# Processing filenames with spaces from a list
cat file_list.txt | xargs -d '\n' mv -t target_directory/

# Handling multi-line commit messages in git
git log --pretty=format:"%s" | xargs -d '\n' -n1 git show

When processing large datasets:

  • -d '\n' is generally faster than shell loops
  • Batch processing with -n can improve efficiency
  • For massive files, combining with parallel may help