How to Convert Line Breaks Between Windows (CRLF) and Unix/Linux (LF) Formats in Text Files


2 views

Working across different operating systems often introduces subtle formatting issues, especially with text files. The most common problem developers face is line ending differences:

  • Windows uses CRLF (Carriage Return + Line Feed: \r\n)
  • Unix/Linux uses LF only (\n)
  • Old Mac systems used CR only (\r)

These invisible differences can cause:

  • Version control conflicts (Git may flag entire files as changed)
  • Script execution failures
  • Unexpected behavior in parsers and processors

Using dos2unix/unix2dos

The most straightforward solution:

# Convert Windows to Unix
dos2unix filename.txt

# Convert Unix to Windows
unix2dos filename.txt

If these commands aren't available, install them:

# On Debian/Ubuntu
sudo apt-get install dos2unix

# On RHEL/CentOS
sudo yum install dos2unix

Using sed (When Standard Tools Aren't Available)

For Windows to Unix conversion:

sed -i 's/\r$//' filename.txt

For Unix to Windows conversion:

sed -i 's/$/\r/' filename.txt

Using tr Command

Simple but less flexible:

# Windows to Unix
tr -d '\r' < windows.txt > unix.txt

# Unix to Windows (requires more work)
awk '{printf "%s\r\n", $0}' unix.txt > windows.txt

For batch processing:

# Convert all .txt files in directory
find . -type f -name "*.txt" -exec dos2unix {} \;

Python Implementation

def convert_line_endings(input_file, output_file, mode):
    with open(input_file, 'rb') as f:
        content = f.read()
    
    if mode == 'unix':
        content = content.replace(b'\r\n', b'\n')
    elif mode == 'windows':
        content = content.replace(b'\n', b'\r\n')
    
    with open(output_file, 'wb') as f:
        f.write(content)

Node.js Implementation

const fs = require('fs');

function convertLineEndings(filePath, target) {
    let content = fs.readFileSync(filePath, 'utf8');
    
    if (target === 'unix') {
        content = content.replace(/\r\n/g, '\n');
    } else if (target === 'windows') {
        content = content.replace(/\n/g, '\r\n');
    }
    
    fs.writeFileSync(filePath, content);
}

Git can handle line endings automatically. Configure it with:

git config --global core.autocrlf true  # Windows
git config --global core.autocrlf input # Linux/Mac

For consistent behavior across teams, consider adding a .gitattributes file:

* text=auto
*.sh text eol=lf
*.bat text eol=crlf

Most modern editors can handle line endings:

  • VS Code: Check the status bar or use the command palette
  • Sublime Text: View → Line Endings menu
  • Notepad++: Edit → EOL Conversion menu

When working across different operating systems, you'll encounter two primary line ending formats:

  • Windows (CRLF): Carriage Return + Line Feed (\r\n)
  • Unix/Linux (LF): Line Feed only (\n)

Here are the most efficient terminal commands for conversion:

# Windows to Unix conversion
sed -i 's/\r$//' filename.txt

# Unix to Windows conversion
sed -i 's/$/\r/' filename.txt

For batch processing multiple files:

# Recursive conversion of all .txt files
find . -type f -name "*.txt" -exec sed -i 's/\r$//' {} +

These specialized tools provide the most robust solution:

# Install on Debian/Ubuntu
sudo apt install dos2unix

# Windows to Unix
dos2unix filename.txt

# Unix to Windows
unix2dos filename.txt

Python Solution

# Convert CRLF to LF
with open('file.txt', 'r', newline='') as f:
    content = f.read()
with open('file.txt', 'w', newline='\n') as f:
    f.write(content)

Node.js Implementation

const fs = require('fs');

// Convert to Unix format
fs.readFile('file.txt', 'utf8', (err, data) => {
  if (err) throw err;
  const unixContent = data.replace(/\r\n/g, '\n');
  fs.writeFile('file.txt', unixContent, 'utf8');
});

For Git users, you can configure automatic line ending conversion:

# Auto convert to LF on commit, CRLF on checkout
git config --global core.autocrlf true

# Force LF for all repositories
git config --global core.eol lf

When dealing with files containing both formats:

# Using awk for selective conversion
awk '{ sub(/\r$/, ""); print }' mixedfile.txt > cleanfile.txt

For large files (GBs in size), consider these optimized methods:

# Using tr for high-speed conversion
tr -d '\r' < windows.txt > unix.txt

# Using perl for in-place conversion
perl -pi -e 's/\r\n/\n/g' largefile.csv
  • VS Code: Click the "CRLF" or "LF" indicator in the status bar
  • Notepad++: Edit → EOL Conversion menu
  • Sublime Text: View → Line Endings menu