How to Create a Single Linux VM Across Multiple Physical Machines Using Distributed Resource Aggregation


2 views

Creating a unified virtual machine that spans across multiple physical servers is essentially a form of distributed virtualization. This approach differs from traditional cluster computing in that it presents a single system image (SSI) to the user, while leveraging resources from multiple nodes behind the scenes.

Several technologies enable this architecture:

1. OpenMosix / LinuxPMI

While no longer actively maintained, OpenMosix demonstrated how to create a single-system image cluster:


# Sample OpenMosix configuration
echo "192.168.1.10 node1" >> /etc/hosts
echo "192.168.1.11 node2" >> /etc/hosts
mosix autoconfig

2. Kerrighed SSI Cluster

A more modern approach using Linux kernel modifications:


# Installing Kerrighed
apt-get install kerrighed-sources
make-kpkg --rootcmd fakeroot --initrd kernel_image
dpkg -i linux-image-*.deb

3. Virtual Machine Distributed Platforms

Commercial solutions like VMware vSphere with Distributed Resource Scheduler (DRS) can achieve similar results:


# Example vSphere PowerCLI command
New-VM -Name "DistributedVM" -ResourcePool Cluster1 -Datastore DS1

For a more contemporary solution, OpenStack's Cells v2 architecture allows aggregating resources:


# Configure Nova cells
nova-manage cell_v2 simple_cell_setup --transport-url rabbit://stackrabbit:password@controller
nova-manage cell_v2 discover_hosts --verbose

When implementing distributed VM solutions, consider:

  • Network latency between nodes
  • Memory coherence protocols
  • Distributed storage performance
  • Failure domain isolation

For specific use cases, you might consider:


# Kubernetes Virtual Kubelet example
kubectl apply -f https://raw.githubusercontent.com/virtual-kubelet/virtual-kubelet/master/examples/kubernetes/provider.yaml

Or using Apache Mesos:


mesos-execute --master=zk://192.168.1.10:2181/mesos --name="dist-vm" --command="uname -a"

Creating a unified virtual machine that spans multiple physical servers involves leveraging distributed computing paradigms with virtualization technologies. This approach combines resources like CPU, RAM, and storage from multiple machines into a single logical entity.

Several proven methods exist for this architecture:


// Example using libvirt for distributed VM management
virsh -c qemu+ssh://host2/system list --all
virsh -c qemu+ssh://host3/system list --all

Linux kernel projects provide foundational support:

  • OpenMosix (historical but conceptually relevant)
  • Linux VServers
  • Kubernetes with Kata Containers

OpenStack Nova can manage VMs across multiple compute nodes:


# nova.conf configuration snippet
[DEFAULT]
compute_driver=libvirt.LibvirtDriver
enable_instances_poll=False

High-speed low-latency networking is critical for performance:


# RDMA configuration example
ibstat
ibv_devinfo

For specific use cases:

  1. MPI-based solutions for HPC workloads
  2. Ganeti for cluster management
  3. Proxmox VE with Ceph storage

Watch for these challenges:


# Check distributed resource visibility
lscpu --extended
free -h
df -h