How to Fix MySQL System Table Corruption (Event Scheduler Damaged Tables Error 1577)


7 views

When you see ERROR 1577 (HY000): Cannot proceed because system tables used by Event Scheduler were found damaged at server start, this typically occurs after an unclean MySQL shutdown (like killing the mysqld process) or disk issues. The Event Scheduler relies on system tables in the mysql database, particularly:

mysql.event
mysql.servers
mysql.time_zone*

First attempt - Run MySQL in recovery mode:

mysqld --skip-grant-tables --skip-networking --event-scheduler=DISABLED

Then connect and repair:

mysqlcheck --repair --databases mysql
mysql_upgrade --force

If automatic repair fails, try this sequence:

# Stop MySQL properly if possible
sudo systemctl stop mysql

# Backup your data directory first!
sudo cp -R /var/lib/mysql /var/lib/mysql_backup

# Rebuild event table structure
mysql -e "DROP TABLE IF EXISTS mysql.event;
          CREATE TABLE mysql.event (
            db CHAR(64) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '',
            name CHAR(64) NOT NULL DEFAULT '',
            body longblob NOT NULL,
            /* ... full table structure ... */
          ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='Events';"

# Run the upgrade
mysql_upgrade --force
  • Always use SET GLOBAL event_scheduler = OFF before major operations
  • Configure innodb_force_recovery in my.cnf when suspecting corruption
  • Regularly check table status with CHECK TABLE mysql.event

Here's how we fixed this on an Ubuntu 20.04 server:

# First identify the corrupted tables
sudo mysqlcheck -u root -p --all-databases --check

# Then for each corrupted table:
sudo mysql -u root -p -e "REPAIR TABLE mysql.event USE_FRM;"

# Final verification
sudo mysql -u root -p -e "SHOW TABLES FROM mysql LIKE 'event';"

When MySQL processes are terminated abruptly (especially during write operations), it can lead to corruption in system tables. The Event Scheduler's metadata tables (mysql.event and related system tables) are particularly vulnerable because they're frequently accessed but not always properly closed during crashes.

First verify the extent of corruption by running:

mysqlcheck --all-databases --check-upgrade --auto-repair
mysqlcheck mysql --repair

If these fail with InnoDB errors, we'll need deeper repairs.

1. Start MySQL in Recovery Mode

Add to your my.cnf/my.ini:

[mysqld]
innodb_force_recovery = 1
skip-grant-tables
event_scheduler=OFF

Then start MySQL and connect:

mysql -u root

2. Repair Core Tables

Execute these critical repairs:

USE mysql;
REPAIR TABLE event;
REPAIR TABLE time_zone;
REPAIR TABLE time_zone_leap_second;
REPAIR TABLE time_zone_name;
REPAIR TABLE time_zone_transition;
REPAIR TABLE time_zone_transition_type;

3. Rebuild System Tables (If Needed)

For severe corruption:

mysql_install_db --user=mysql --ldata=/var/lib/mysql

Configuration Recommendations

Add these to your my.cnf:

[mysqld]
innodb_flush_log_at_trx_commit = 1
sync_binlog = 1
event_scheduler = ON

Monitoring Script

Create a watchdog script:

#!/bin/bash
if ! mysqladmin ping &>/dev/null; then
    systemctl stop mysql
    mysqlcheck --all-databases --repair
    systemctl start mysql
fi