How to Convert Line Breaks Between Windows (CRLF) and Unix/Linux (LF) Formats in Text Files


21 views

Working across different operating systems often introduces subtle formatting issues, especially with text files. The most common problem developers face is line ending differences:

  • Windows uses CRLF (Carriage Return + Line Feed: \r\n)
  • Unix/Linux uses LF only (\n)
  • Old Mac systems used CR only (\r)

These invisible differences can cause:

  • Version control conflicts (Git may flag entire files as changed)
  • Script execution failures
  • Unexpected behavior in parsers and processors

Using dos2unix/unix2dos

The most straightforward solution:

# Convert Windows to Unix
dos2unix filename.txt

# Convert Unix to Windows
unix2dos filename.txt

If these commands aren't available, install them:

# On Debian/Ubuntu
sudo apt-get install dos2unix

# On RHEL/CentOS
sudo yum install dos2unix

Using sed (When Standard Tools Aren't Available)

For Windows to Unix conversion:

sed -i 's/\r$//' filename.txt

For Unix to Windows conversion:

sed -i 's/$/\r/' filename.txt

Using tr Command

Simple but less flexible:

# Windows to Unix
tr -d '\r' < windows.txt > unix.txt

# Unix to Windows (requires more work)
awk '{printf "%s\r\n", $0}' unix.txt > windows.txt

For batch processing:

# Convert all .txt files in directory
find . -type f -name "*.txt" -exec dos2unix {} \;

Python Implementation

def convert_line_endings(input_file, output_file, mode):
    with open(input_file, 'rb') as f:
        content = f.read()
    
    if mode == 'unix':
        content = content.replace(b'\r\n', b'\n')
    elif mode == 'windows':
        content = content.replace(b'\n', b'\r\n')
    
    with open(output_file, 'wb') as f:
        f.write(content)

Node.js Implementation

const fs = require('fs');

function convertLineEndings(filePath, target) {
    let content = fs.readFileSync(filePath, 'utf8');
    
    if (target === 'unix') {
        content = content.replace(/\r\n/g, '\n');
    } else if (target === 'windows') {
        content = content.replace(/\n/g, '\r\n');
    }
    
    fs.writeFileSync(filePath, content);
}

Git can handle line endings automatically. Configure it with:

git config --global core.autocrlf true  # Windows
git config --global core.autocrlf input # Linux/Mac

For consistent behavior across teams, consider adding a .gitattributes file:

* text=auto
*.sh text eol=lf
*.bat text eol=crlf

Most modern editors can handle line endings:

  • VS Code: Check the status bar or use the command palette
  • Sublime Text: View → Line Endings menu
  • Notepad++: Edit → EOL Conversion menu

When working across different operating systems, you'll encounter two primary line ending formats:

  • Windows (CRLF): Carriage Return + Line Feed (\r\n)
  • Unix/Linux (LF): Line Feed only (\n)

Here are the most efficient terminal commands for conversion:

# Windows to Unix conversion
sed -i 's/\r$//' filename.txt

# Unix to Windows conversion
sed -i 's/$/\r/' filename.txt

For batch processing multiple files:

# Recursive conversion of all .txt files
find . -type f -name "*.txt" -exec sed -i 's/\r$//' {} +

These specialized tools provide the most robust solution:

# Install on Debian/Ubuntu
sudo apt install dos2unix

# Windows to Unix
dos2unix filename.txt

# Unix to Windows
unix2dos filename.txt

Python Solution

# Convert CRLF to LF
with open('file.txt', 'r', newline='') as f:
    content = f.read()
with open('file.txt', 'w', newline='\n') as f:
    f.write(content)

Node.js Implementation

const fs = require('fs');

// Convert to Unix format
fs.readFile('file.txt', 'utf8', (err, data) => {
  if (err) throw err;
  const unixContent = data.replace(/\r\n/g, '\n');
  fs.writeFile('file.txt', unixContent, 'utf8');
});

For Git users, you can configure automatic line ending conversion:

# Auto convert to LF on commit, CRLF on checkout
git config --global core.autocrlf true

# Force LF for all repositories
git config --global core.eol lf

When dealing with files containing both formats:

# Using awk for selective conversion
awk '{ sub(/\r$/, ""); print }' mixedfile.txt > cleanfile.txt

For large files (GBs in size), consider these optimized methods:

# Using tr for high-speed conversion
tr -d '\r' < windows.txt > unix.txt

# Using perl for in-place conversion
perl -pi -e 's/\r\n/\n/g' largefile.csv
  • VS Code: Click the "CRLF" or "LF" indicator in the status bar
  • Notepad++: Edit → EOL Conversion menu
  • Sublime Text: View → Line Endings menu