Working across different operating systems often introduces subtle formatting issues, especially with text files. The most common problem developers face is line ending differences:
- Windows uses CRLF (Carriage Return + Line Feed: \r\n)
- Unix/Linux uses LF only (\n)
- Old Mac systems used CR only (\r)
These invisible differences can cause:
- Version control conflicts (Git may flag entire files as changed)
- Script execution failures
- Unexpected behavior in parsers and processors
Using dos2unix/unix2dos
The most straightforward solution:
# Convert Windows to Unix
dos2unix filename.txt
# Convert Unix to Windows
unix2dos filename.txt
If these commands aren't available, install them:
# On Debian/Ubuntu
sudo apt-get install dos2unix
# On RHEL/CentOS
sudo yum install dos2unix
Using sed (When Standard Tools Aren't Available)
For Windows to Unix conversion:
sed -i 's/\r$//' filename.txt
For Unix to Windows conversion:
sed -i 's/$/\r/' filename.txt
Using tr Command
Simple but less flexible:
# Windows to Unix
tr -d '\r' < windows.txt > unix.txt
# Unix to Windows (requires more work)
awk '{printf "%s\r\n", $0}' unix.txt > windows.txt
For batch processing:
# Convert all .txt files in directory
find . -type f -name "*.txt" -exec dos2unix {} \;
Python Implementation
def convert_line_endings(input_file, output_file, mode):
with open(input_file, 'rb') as f:
content = f.read()
if mode == 'unix':
content = content.replace(b'\r\n', b'\n')
elif mode == 'windows':
content = content.replace(b'\n', b'\r\n')
with open(output_file, 'wb') as f:
f.write(content)
Node.js Implementation
const fs = require('fs');
function convertLineEndings(filePath, target) {
let content = fs.readFileSync(filePath, 'utf8');
if (target === 'unix') {
content = content.replace(/\r\n/g, '\n');
} else if (target === 'windows') {
content = content.replace(/\n/g, '\r\n');
}
fs.writeFileSync(filePath, content);
}
Git can handle line endings automatically. Configure it with:
git config --global core.autocrlf true # Windows
git config --global core.autocrlf input # Linux/Mac
For consistent behavior across teams, consider adding a .gitattributes
file:
* text=auto
*.sh text eol=lf
*.bat text eol=crlf
Most modern editors can handle line endings:
- VS Code: Check the status bar or use the command palette
- Sublime Text: View → Line Endings menu
- Notepad++: Edit → EOL Conversion menu
When working across different operating systems, you'll encounter two primary line ending formats:
- Windows (CRLF): Carriage Return + Line Feed (\r\n)
- Unix/Linux (LF): Line Feed only (\n)
Here are the most efficient terminal commands for conversion:
# Windows to Unix conversion
sed -i 's/\r$//' filename.txt
# Unix to Windows conversion
sed -i 's/$/\r/' filename.txt
For batch processing multiple files:
# Recursive conversion of all .txt files
find . -type f -name "*.txt" -exec sed -i 's/\r$//' {} +
These specialized tools provide the most robust solution:
# Install on Debian/Ubuntu
sudo apt install dos2unix
# Windows to Unix
dos2unix filename.txt
# Unix to Windows
unix2dos filename.txt
Python Solution
# Convert CRLF to LF
with open('file.txt', 'r', newline='') as f:
content = f.read()
with open('file.txt', 'w', newline='\n') as f:
f.write(content)
Node.js Implementation
const fs = require('fs');
// Convert to Unix format
fs.readFile('file.txt', 'utf8', (err, data) => {
if (err) throw err;
const unixContent = data.replace(/\r\n/g, '\n');
fs.writeFile('file.txt', unixContent, 'utf8');
});
For Git users, you can configure automatic line ending conversion:
# Auto convert to LF on commit, CRLF on checkout
git config --global core.autocrlf true
# Force LF for all repositories
git config --global core.eol lf
When dealing with files containing both formats:
# Using awk for selective conversion
awk '{ sub(/\r$/, ""); print }' mixedfile.txt > cleanfile.txt
For large files (GBs in size), consider these optimized methods:
# Using tr for high-speed conversion
tr -d '\r' < windows.txt > unix.txt
# Using perl for in-place conversion
perl -pi -e 's/\r\n/\n/g' largefile.csv
- VS Code: Click the "CRLF" or "LF" indicator in the status bar
- Notepad++: Edit → EOL Conversion menu
- Sublime Text: View → Line Endings menu