Advanced MySQL Memory Optimization for MyISAM on High-CPU EC2 Instances


2 views

To check MySQL's current memory usage on Linux, use these commands:

# Show system-wide memory usage
free -m

# Check MySQL process memory
ps aux | grep mysqld | grep -v grep

# Detailed memory breakdown (requires pmem)
mysqladmin -uroot -p extended-status | grep -E 'Key_buffer_size|Sort_buffer_size|Read_buffer_size'

For your 7GB EC2 instance running MyISAM, these are critical parameters to tune:

# In my.cnf configuration
[mysqld]
key_buffer_size = 2G         # 25-30% of total RAM for MyISAM
myisam_sort_buffer_size = 256M
read_buffer_size = 4M
read_rnd_buffer_size = 4M
sort_buffer_size = 8M
join_buffer_size = 8M
thread_stack = 256K
thread_cache_size = 16
table_open_cache = 4000

For your 2GB+500MB databases, calculate optimal settings:

SELECT 
  CONCAT(ROUND(SUM(index_length)/(1024*1024),2),'MB') AS 'Index Size',
  CONCAT(ROUND(SUM(data_length)/(1024*1024),2),'MB') AS 'Data Size'
FROM information_schema.TABLES 
WHERE engine='MyISAM';

Then adjust key_buffer_size to be slightly larger than your total index size.

Set up these monitoring checks:

# Key buffer efficiency
SHOW STATUS LIKE 'key_read%';

# Cache hit ratio formula
SELECT (1-(Value2/Value1))*100 AS cache_hit_ratio
FROM 
  (SELECT variable_value AS Value1 
   FROM information_schema.global_status 
   WHERE variable_name='Key_read_requests') AS A,
  (SELECT variable_value AS Value2 
   FROM information_schema.global_status 
   WHERE variable_name='Key_reads') AS B;

With 8 virtual cores:

innodb_io_capacity = 2000
innodb_read_io_threads = 8
innodb_write_io_threads = 8
innodb_buffer_pool_instances = 8

For your specific EC2 instance:

[mysqld]
# Memory Settings
key_buffer_size = 3G
myisam_sort_buffer_size = 384M
query_cache_size = 256M
query_cache_limit = 4M
tmp_table_size = 256M
max_heap_table_size = 256M

# Connection Settings
max_connections = 200
thread_cache_size = 32
table_open_cache = 4096
table_definition_cache = 4096

To check MySQL's current memory utilization on Linux, use these commands:

# Overall system memory usage
free -m

# MySQL-specific memory allocation
ps aux | grep mysqld
pmap -x $(pgrep mysqld)

# Show MySQL system variables
mysql -e "SHOW VARIABLES LIKE '%buffer%';"
mysql -e "SHOW VARIABLES LIKE '%cache%';"

For detailed monitoring, install and configure MySQLTuner:

wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl
chmod +x mysqltuner.pl
./mysqltuner.pl

For a 7GB EC2 instance running MyISAM databases, consider these settings in your my.cnf:

[mysqld]
key_buffer_size = 2G        # 25-30% of total RAM for MyISAM indexes
myisam_sort_buffer_size = 256M
read_buffer_size = 4M
read_rnd_buffer_size = 4M
sort_buffer_size = 4M
join_buffer_size = 4M
thread_cache_size = 16
table_open_cache = 2048
query_cache_size = 128M     # If using query caching

Consider these additional optimizations:

# For frequently accessed tables, preload indexes:
LOAD INDEX INTO CACHE db1.table1, db1.table2;

# Monitor key buffer efficiency:
mysql> SHOW STATUS LIKE 'key%';
+------------------------+-----------+
| Variable_name          | Value     |
+------------------------+-----------+
| Key_blocks_unused      | 102400    |
| Key_read_requests      | 58324567  |
| Key_reads              | 5         |
+------------------------+-----------+

Calculate key buffer efficiency:

Key_buffer_efficiency = (1 - Key_reads/Key_read_requests) * 100

While optimizing MyISAM:

  • Run OPTIMIZE TABLE monthly on frequently updated tables
  • Consider partitioning large tables
  • For read-heavy workloads, increase key_buffer_size further
  • For write-heavy workloads, consider myisam_max_sort_file_size

Example partitioning code:

ALTER TABLE large_table 
PARTITION BY RANGE (YEAR(created_at)) (
    PARTITION p2020 VALUES LESS THAN (2021),
    PARTITION p2021 VALUES LESS THAN (2022),
    PARTITION p2022 VALUES LESS THAN (2023),
    PARTITION pmax VALUES LESS THAN MAXVALUE
);

For your High CPU XL instance:

innodb_io_capacity = 2000    # For EBS volumes
innodb_flush_neighbors = 0   # For SSD storage
innodb_read_io_threads = 8
innodb_write_io_threads = 8