Configuring Multiple Python Versions (2.7/3.x) Under Single uWSGI Emperor with Plugin System


4 views

When managing multiple Python applications with different version requirements under a single uWSGI Emperor setup, we face interpreter conflicts. The fundamental issue arises when:

  • Main uWSGI binary compiled with Python 2.7
  • Vassals require Python 3.x virtualenvs
  • Traditional virtualenv activation fails with ImportError: No module named site

The most robust approach involves compiling uWSGI with modular Python support. Here's the complete workflow:

Step 1: Build uWSGI with Plugin Support

# Install development dependencies
sudo apt-get install build-essential python3-dev python2.7-dev

# Download uWSGI source
wget https://projects.unbit.it/downloads/uwsgi-latest.tar.gz
tar xzvf uwsgi-latest.tar.gz
cd uwsgi-*/

# Build with both Python versions
make PROFILE=nolang
python2.7 uwsgiconfig.py --plugin plugins/python python27
python3.4 uwsgiconfig.py --plugin plugins/python python34

Step 2: Emperor Configuration

Modify your emperor startup script (/etc/rc.local):

#!/bin/bash
/usr/local/bin/uwsgi \
    --emperor /etc/uwsgi/vassals \
    --uid www-data \
    --gid www-data \
    --daemonize /var/log/uwsgi/emperor.log \
    --plugins python27,python34

For Python 3.4 application:

[uwsgi]
plugin = python34
virtualenv = /path/to/python34_venv
wsgi-file = project/wsgi.py

For Python 2.7 application:

[uwsgi]
plugin = python27
virtualenv = /path/to/python27_venv
wsgi-file = project/wsgi.py

For production environments, consider these optimizations:

[uwsgi]
# Python 3.5+ specific configuration
plugin = python35
enable-threads = true
thunder-lock = true
harakiri = 30
max-requests = 1000

Check loaded plugins with:

uwsgi --plugins-list

Confirm Python version in vassal logs:

print(sys.version)
  • Each plugin adds ~5MB memory overhead
  • Plugin loading occurs at vassal startup
  • No cross-interpreter communication penalty

When managing multiple Python applications through uWSGI's Emperor mode, a common pain point emerges when vassals require different Python versions. The fundamental issue stems from uWSGI being compiled against a specific Python version (Python 2.7 in your case), which creates interpreter conflicts when attempting to run Python 3 virtualenvs.

The most maintainable approach involves compiling uWSGI with multiple Python interpreter plugins. Here's how the architecture works:

uWSGI Core (language-independent)
├── Python 2.7 plugin (libpython2.7.so)
├── Python 3.4 plugin (libpython3.4m.so)
└── Python 3.6 plugin (libpython3.6m.so)

First, build uWSGI with embedded Python support disabled and plugins enabled:

UWSGI_PROFILE=core make
python3.4 uwsgiconfig.py --plugin plugins/python python34
python2.7 uwsgiconfig.py --plugin plugins/python python27

This creates plugin shared objects (.so files) in your build directory.

For a Python 3.4 application:

[uwsgi]
plugin = python34
virtualenv = /path/to/python3env
wsgi-file = app3/wsgi.py

For a Python 2.7 application:

[uwsgi]
plugin = python27
virtualenv = /path/to/python2env
wsgi-file = app2/wsgi.py

To confirm your plugins are loading correctly, add this to your vassal config:

[uwsgi]
...
plugins-dir = /usr/lib/uwsgi/plugins
logger = file:/var/log/uwsgi/plugin-debug.log

Check the log for messages like:

*** Starting uWSGI 2.0.20 (64bit) on [...
*** Python version: 3.4.10 (default, ...)

When running mixed Python versions:

  • Each plugin maintains its own GIL state
  • Memory overhead is minimal (~5MB per interpreter)
  • Consider using lazy-apps = true for better resource utilization

To avoid repeating plugin paths in every vassal:

[emperor]
plugins-dir = /usr/local/lib/uwsgi
emperor-tyrant = true

This ensures all vassals inherit the plugin directory setting while maintaining their individual interpreter requirements.