How to Remove Every Other Newline in Bash Output Using sed/awk: A Concise Guide for Linux Systems


2 views

When working with CLI tools in Linux, we often encounter output formats where related data appears on separate lines. Consider this common pattern from storage system outputs:

Runtime Name: vmhba2:C0:T3:L14
Group State: active
Runtime Name: vmhba3:C0:T0:L14
Group State: active unoptimized

The naive approach using tr has significant limitations:

# Problematic approach:
command | tr "\\nGroup" " "

This fails because it:
1. Removes ALL newlines
2. Unintentionally replaces characters from the word "Group"
3. Doesn't preserve the logical record structure

Using sed

The stream editor provides the cleanest solution:

command | sed 'N;s/\n/ /'

Explanation:
- N appends the next line to pattern space
- s/\n/ / replaces the newline with a space

awk Approach

For more complex transformations, awk offers better control:

command | awk 'NR%2{printf "%s ",$0;next}1'

How it works:
- NR%2 matches odd-numbered lines
- printf prints without newline
- next skips to next line
- 1 is awk's shorthand for print

For production scripts, consider these enhancements:

# Preserve trailing newline
command | sed -e ':a' -e 'N;s/\n/ /;ta'

# Handle empty lines
command | awk 'NF{printf "%s%s",(NR%2?$0" ":x),(!(NR%2)?RS:x)}'

For large files (10k+ lines):

# Fastest method (GNU sed)
command | sed -z 's/\n$Group$/ \1/g'

# Portable version
command | paste -d " " - -

When working with CLI tools in Linux, we often encounter output formats where related information spans multiple lines. In this case, we have VM runtime information where each record uses two lines:

Runtime Name: vmhba2:C0:T3:L14
Group State: active
Runtime Name: vmhba3:C0:T0:L14
Group State: active unoptimized

The initial attempt using tr fails because:

... | tr "\nGroup" " "

This command replaces ALL newlines and also unexpectedly converts characters 'G','r','o','u','p' to spaces.

Using sed

This is the most straightforward solution:

sed 'N;s/\n/ /'

How it works:
- N appends the next line to pattern space
- s/\n/ / replaces the newline with a space

Using awk

For more complex processing:

awk '{printf "%s", $0; if (NR%2==0) print ""; else printf " "}'

Or alternatively:

awk 'ORS=NR%2?" ":"\n"'

Using paste (for fixed 2-line records)

paste -d " " - -

If your input might have odd number of lines:

sed '$!N;s/\n/ /'

The $! prevents reading next line when on last line.

Here's how to process vSphere CLI output:

esxcli storage nmp device list | 
awk '/Runtime Name:|Group State:/ {if (++i%2) {printf "%s ", $0} else print $0}'

For large files:
- sed is generally fastest
- awk solutions offer more flexibility
- paste is fastest for simple cases but least flexible