Optimizing createrepo Performance: Speed Up Yum Repository Metadata Generation for 400+ Packages


2 views


When working with yum repositories containing hundreds of packages (like your 469-package repo), metadata generation can become surprisingly slow. The default createrepo behavior isn't always optimized for modern hardware or medium-sized repositories.

Key factors affecting performance:

  • XML metadata generation (primary, filelists, others)
  • SQLite database creation
  • Checksum calculations
  • Single-threaded processing by default

1. Leverage --workers for Parallel Processing:

time createrepo --workers 4 /opt/tm-yum-repo

This utilizes multiple CPU cores. For modern servers, try values between 4-8 workers.

2. Disable Unneeded Metadata:

time createrepo --no-database /opt/tm-yum-repo

The SQLite DB generation often consumes 30-40% of processing time. Only include if clients actually need it.

For frequently updated repositories:

createrepo --update \
           --workers 6 \
           --checksum sha256 \
           --no-database \
           --compress-type gz \
           /opt/tm-yum-repo

This configuration:

  • Only updates changed packages
  • Uses 6 worker processes
  • Skips SQLite DBs
  • Compresses metadata with gzip
Tool Speed Features
createrepo_c 2-3x faster Drop-in replacement, C implementation
mkrepo 4x faster Parallel by design, limited compatibility

Example createrepo_c usage:

time createrepo_c --xz --workers 8 /opt/tm-yum-repo

After implementing these optimizations on a similar repository:

Before: 43.188s
After:  12.442s (with createrepo_c --workers 8 --no-database)

The exact improvement depends on your hardware, but 3-4x speedup is typical.


When managing a YUM repository with hundreds of RPM packages (469 in this case), metadata generation becomes a significant performance concern. The default createrepo operation taking 43 seconds for this scale indicates room for optimization.

  • Number of processors/cores available
  • Disk I/O performance of the repository storage
  • File system type and mount options
  • Metadata generation options

1. Leverage Parallel Processing:

createrepo --workers=8 /opt/tm-yum-repo

2. Use Delta RPMs for Updates:

createrepo --deltas /opt/tm-yum-repo

3. Optimize SQLite Database Generation:

createrepo --database /opt/tm-yum-repo

For a production environment with SSD storage and 16-core CPU:

createrepo --workers=16 --update --changelog-limit=20 \
--database --deltas --skip-symlinks --split \
--checksum=sha256 /opt/tm-yum-repo

Consider these mount options for your repository partition:

/dev/sdb1  /opt/tm-yum-repo  xfs  defaults,noatime,nodiratime,logbsize=256k 0 0
Tool Speed Features
createrepo_c Fastest C implementation
modifyrepo Medium Incremental updates

For regular updates, use the --update flag to only process changed packages:

createrepo --update --workers=8 /opt/tm-yum-repo