Debugging and Fixing CentOS LXC Container Startup Failures on Ubuntu Hosts


4 views

The log reveals several key issues during container initialization:

lxc-start 1416596263.031 ERROR    lxc_apparmor - lsm/apparmor.c:mount_feature_enabled:61 - Permission denied - Error mounting securityfs
lxc-start 1416596263.032 WARN     lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:184 - Incomplete AppArmor support in your kernel

Based on the log patterns, we're seeing multiple potential failure points:

  • AppArmor security policy conflicts
  • Potential issues with the host's cgroup configuration
  • System call filtering problems (visible in seccomp warnings)

First, let's verify the container configuration with this command:

lxc-checkconfig

For a more detailed analysis, run the container in foreground mode:

lxc-start -n container_name -F --logpriority=DEBUG

Try these configuration adjustments in your container's config file (/var/lib/lxc/stash/config):

# Disable AppArmor profile if causing issues
lxc.aa_profile = unconfined

# Alternative cgroup management
lxc.cgroup.use = cgmanager

# Additional capabilities if needed
lxc.cap.drop = mac_admin mac_override sys_time

Ensure required kernel modules are loaded:

lsmod | grep -E 'apparmor|bridge|veth|nft'

If missing, load them with:

modprobe apparmor
modprobe bridge

For networking issues, verify bridge setup:

brctl show
ip link show

Example network configuration snippet:

lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up

For persistent issues, strace can reveal deeper problems:

strace -f -o container-strace.log lxc-start -n container_name -F

Key areas to examine in the strace output:

  • Failed system calls (returning -1)
  • Permission denied errors (EACCES)
  • Missing file errors (ENOENT)

If standard start fails, try the direct init approach:

lxc-execute -n container_name -- /sbin/init

Or for minimal debugging:

lxc-start -n container_name -- /bin/bash

Ubuntu 14.10 with CentOS 6.5 containers may require specific lxc versions. Verify packages:

dpkg -l | grep lxc
yum --installroot=/var/lib/lxc/stash/rootfs list installed

When attempting to launch CentOS 6.5 LXC containers on an Ubuntu 14.10 host system, we encounter immediate failure during initialization. The log reveals two primary suspects:

lxc-start 1416596263.031 ERROR    lxc_apparmor - lsm/apparmor.c:mount_feature_enabled:61 - Permission denied - Error mounting securityfs
lxc-start 1416596263.032 WARN     lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:184 - Incomplete AppArmor support in your kernel

The container fails during the security initialization phase, specifically with:

  1. AppArmor profile loading issues
  2. Seccomp filter complications (particularly with finit_module syscall)
  3. Potential cgroup permissions problems

First, verify the host's security subsystem status:

# Check AppArmor status
sudo apparmor_status
# Verify kernel seccomp support
grep CONFIG_SECCOMP= /boot/config-$(uname -r)

For temporary testing, you can disable AppArmor profile enforcement:

# Edit container configuration
sudo nano /var/lib/lxc/stash/config
# Add these directives:
lxc.aa_profile = unconfined
lxc.aa_allow_incomplete = 1

The log shows particular trouble with finit_module syscall handling. Modify your seccomp policy:

lxc.seccomp = /var/lib/lxc/stash/seccomp.conf

Sample seccomp configuration:

# Basic seccomp policy
[default]
# Whitelist common syscalls
allow

The container successfully initializes network interfaces but fails later. Check with:

lxc-start -n stash -F --logpriority=DEBUG

Ensure required modules are loaded:

sudo modprobe overlay
sudo modprobe veth
sudo modprobe nf_nat

Try launching with reduced security restrictions for testing:

lxc-start -n stash -s 'lxc.seccomp = none' -s 'lxc.aa_profile = unconfined'

For production environments, create a custom AppArmor profile:

#include 
profile lxc-container-stash flags=(attach_disconnected,mediate_deleted) {
  # Add container-specific rules here
}

Key package versions to verify:

# On Ubuntu host
dpkg -l | grep -E 'lxc|apparmor'
# In CentOS container (if accessible)
rpm -qa | grep -E 'systemd|initscripts'

If issues persist, consider container migration approaches:

# Export container configuration
lxc-config -n stash > stash.conf.backup
# Attempt conversion to LXD
lxd-migrate -s /var/lib/lxc/stash