When attempting to start MySQL with --skip-grant-tables for database recovery, many users encounter the frustrating error about ibdata1 being unwritable. This typically stems from permission issues or improper shutdown procedures.

• After abrupt MySQL service termination
• When modifying MySQL data directory permissions
• During attempts to bypass security for database recovery
• After OS updates that reset file permissions

1. Verify File Ownership

sudo ls -la /var/lib/mysql/ibdata1
# Expected output should show mysql:mysql ownership
# If not, correct with:
sudo chown mysql:mysql /var/lib/mysql/ibdata1

2. Check Directory Permissions

sudo ls -ld /var/lib/mysql
# Should show drwxr-x--- mysql mysql
# Fix if needed:
sudo chmod 750 /var/lib/mysql

3. Validate AppArmor/SELinux Configuration

For Ubuntu systems:

sudo aa-status | grep mysql
# If AppArmor is blocking, either:
sudo aa-complain /usr/sbin/mysqld
# Or permanently add exceptions:
sudo nano /etc/apparmor.d/local/usr.sbin.mysqld

4. Safe Startup Procedure

To properly start MySQL in recovery mode:

sudo systemctl stop mysql
sudo mysqld_safe --skip-grant-tables --skip-networking &

When Standard Fixes Fail

If permissions appear correct but the error persists:

# Check for filesystem errors
sudo fsck /dev/sdX
# Verify inode usage
df -i /var/lib/mysql

MySQL Configuration Checks

Review your my.cnf for potential conflicts:

[mysqld]
innodb_data_home_dir = /var/lib/mysql
innodb_data_file_path = ibdata1:12M:autoextend
# Ensure no duplicate entries exist

• Always stop MySQL properly with sudo systemctl stop mysql
• Maintain regular backups of your ibdata1 file
• Document any permission changes made to MySQL directories
• Consider using mysql_upgrade instead of manual database imports

Here's how I recovered a production server with similar issues:

# First, verify no MySQL processes are running
sudo pkill -9 mysqld

# Then reset ownership recursively
sudo chown -R mysql:mysql /var/lib/mysql

# Finally start with logging enabled
sudo mysqld_safe --log-error=/var/log/mysql/recovery.log \
--skip-grant-tables --skip-networking

---

When attempting to start MySQL with --skip-grant-tables after improper shutdown procedures, you might encounter the critical InnoDB error indicating the system tablespace file (ibdata1) isn't writable. This typically occurs when:

• MySQL service was improperly terminated
• File permissions were altered during manual configuration
• Disk space is exhausted
• SELinux/AppArmor restrictions are active

Before proceeding with solutions, verify these key points:

# Check file permissions
ls -l /var/lib/mysql/ibdata1

# Verify disk space
df -h /var/lib/mysql

# Check for running processes
ps aux | grep mysql

# Examine error logs
tail -n 50 /var/log/mysql/error.log

1. Permission Repair

The most common fix involves resetting ownership and permissions:

sudo chown -R mysql:mysql /var/lib/mysql
sudo chmod -R 755 /var/lib/mysql
sudo systemctl restart mysql

2. Safe Mode Startup

For recovery situations requiring --skip-grant-tables:

sudo mysqld_safe --skip-grant-tables --skip-networking &
mysql -u root

Once connected, flush privileges and reset root password:

FLUSH PRIVILEGES;
ALTER USER 'root'@'localhost' IDENTIFIED BY 'new_password';

3. Database Recovery

If the ibdata1 file is corrupted:

# Create backup
sudo cp -a /var/lib/mysql /var/lib/mysql_backup

# Clean existing data
sudo rm -rf /var/lib/mysql/*

# Reinitialize data directory
sudo mysqld --initialize --user=mysql

For persistent issues, consider these measures:

# Verify AppArmor/SELinux status
sudo aa-status
sudo sestatus

# Temporary disable AppArmor
sudo systemctl stop apparmor
sudo aa-complain /usr/sbin/mysqld

# Check filesystem integrity
sudo fsck /dev/sdX

• Always stop MySQL service properly: sudo systemctl stop mysql
• Maintain regular backups of /var/lib/mysql
• Monitor disk space with tools like ncdu
• Document all permission changes to MySQL directories