Configuring pfSense with ESXi: Public IP Passthrough and NAT Setup for VMs


2 views

In a typical datacenter setup with ESXi virtualization, we often need to handle both public IP passthrough and NAT configurations simultaneously. The scenario involves:

  • Primary host IP (x.x.x.210) on the physical server
  • Two VMs needing direct public IP access (.211 and .212)
  • Additional VMs requiring NAT through pfSense (.2 and .3 on 192.168.1.0/24)
Network Diagram:
   inet                      lan
[x.x.x.210] -- (NAT) -- [192.168.1.1]
  |-- .211  -- VM1        |--     .2 -- VM3
  \-- .212  -- VM2        \--     .3 -- VM4

First, configure your ESXi vSwitches:

# Create vSwitch0 for WAN (public IPs)
esxcli network vswitch standard add --vswitch-name=vSwitch0
esxcli network vswitch standard uplink add --vswitch-name=vSwitch0 --uplink-name=vmnic0

# Create port group for public IP VMs
esxcli network vswitch standard portgroup add \
--portgroup-name=Public-IPs \
--vswitch-name=vSwitch0

# Create vSwitch1 for LAN (NAT)
esxcli network vswitch standard add --vswitch-name=vSwitch1
esxcli network vswitch standard uplink add --vswitch-name=vSwitch1 --uplink-name=vmnic1

# Create port group for NAT VMs
esxcli network vswitch standard portgroup add \
--portgroup-name=NAT-LAN \
--vswitch-name=vSwitch1

For the pfSense VM installation:

  1. Assign WAN interface to vSwitch0 (Public-IPs port group)
  2. Assign LAN interface to vSwitch1 (NAT-LAN port group)
  3. Configure WAN interface with x.x.x.210
  4. Set up NAT rules for the 192.168.1.0/24 network

Example NAT configuration in pfSense:

# In pfSense web UI (System > Advanced > Firewall & NAT)
1. Navigate to Firewall > NAT > Outbound
2. Select "Hybrid Outbound NAT rule generation"
3. Add manual rule for 192.168.1.0/24 to use WAN address
   - Interface: WAN
   - Source: 192.168.1.0/24
   - NAT Address: WAN address

For VM1 and VM2 needing direct public IPs:

# Sample VMX configuration for public IP VM
ethernet0.present = "TRUE"
ethernet0.connectionType = "custom"
ethernet0.virtualDev = "vmxnet3"
ethernet0.networkName = "Public-IPs"
ethernet0.addressType = "generated"
ethernet0.wakeOnPcktRcv = "FALSE"

Configure static IPs in the guest OS:

# Example for Linux VM (Ubuntu)
network:
  version: 2
  renderer: networkd
  ethernets:
    ens160:
      addresses:
        - x.x.x.211/24
      gateway4: x.x.x.1
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]

To enable DHCP for public IPs through pfSense:

1. Services > DHCP Server > WAN
2. Enable DHCP server
3. Range: x.x.x.211 - x.x.x.220
4. DNS servers: Your preferred public DNS
5. Gateway: x.x.x.1
6. Save and apply changes

Essential firewall rules to maintain security:

# WAN rules (Firewall > Rules > WAN)
1. Allow ICMP (for ping)
2. Allow HTTP/HTTPS if running web services
3. Default deny all other incoming traffic

# LAN rules (Firewall > Rules > LAN)
1. Allow all outbound from LAN
2. Allow specific ports between VMs if needed

If you encounter problems:

  • Check vSwitch port group assignments
  • Verify pfSense interface assignments
  • Test connectivity between NAT and public IP VMs
  • Review firewall logs for blocked traffic

For packet capture debugging:

# In pfSense shell
tcpdump -i em0 -n -v 'host x.x.x.211'

In my datacenter environment, I'm dealing with a server that has multiple public IPs (x.x.x.210 as primary, with .211 and .212 as additional addresses). With the migration to ESXi virtualization, I need to implement this specific network configuration:

Public IP Block:
x.x.x.210 - ESXi Host
x.x.x.211 - VM1 (direct public access)
x.x.x.212 - VM2 (direct public access)

Private Network:
192.168.1.1 - pfSense WAN
192.168.1.2 - VM3 (NATed through pfSense)
192.168.1.3 - VM4 (NATed through pfSense)

First, set up your vSwitches in ESXi:

# Create vSwitch for public IP passthrough
esxcli network vswitch standard add --vswitch-name=vSwitch-Public
esxcli network vswitch standard portgroup add --portgroup-name=PG-Public --vswitch-name=vSwitch-Public

# Create vSwitch for private NAT network
esxcli network vswitch standard add --vswitch-name=vSwitch-Private
esxcli network vswitch standard portgroup add --portgroup-name=PG-Private --vswitch-name=vSwitch-Private

# Assign physical adapters
esxcli network vswitch standard uplink add --uplink-name=vmnic0 --vswitch-name=vSwitch-Public
esxcli network vswitch standard uplink add --uplink-name=vmnic1 --vswitch-name=vSwitch-Private

In pfSense web interface (192.168.1.1):

  1. Navigate to Interfaces > Assign
  2. Create these interfaces:
    • WAN (vmx0) - Configure with ESXi host's main IP (x.x.x.210)
    • LAN (vmx1) - Configure with 192.168.1.1/24
    • OPT1 (optional for future IPv6)

For VMs requiring direct public IPs (VM1 and VM2):

# Example VMX configuration for VM1 (direct public IP)
ethernet0.present = "TRUE"
ethernet0.connectionType = "custom"
ethernet0.virtualDev = "vmxnet3"
ethernet0.networkName = "PG-Public"
ethernet0.addressType = "generated"
ethernet0.pciSlotNumber = "33"

Then configure static IPs in the guest OS:

# Example for Ubuntu VM1
network:
  version: 2
  ethernets:
    ens160:
      addresses: [x.x.x.211/24]
      gateway4: x.x.x.1
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]

For VMs behind NAT (VM3 and VM4):

  1. Navigate to Firewall > NAT > Outbound
  2. Select "Manual Outbound NAT"
  3. Add rules to masquerade private IPs to WAN IP
# Sample pfSense NAT rule (in config.xml)
<rule>
  <source>
    <address>192.168.1.0/24</address>
  </source>
  <destination>
    <any/>
  </destination>
  <protocol>any</protocol>
  <target>x.x.x.210</target>
  <interface>wan</interface>
  <descr><![CDATA[NAT for private VMs]]></descr>
  <associated-rule-id>nat_wan_static</associated-rule-id>
  <static-port>no</static-port>
</rule>

Configure pfSense firewall to protect both public and private VMs:

# WAN rules (pfSense CLI)
pfctl -t public_vms -T add x.x.x.211
pfctl -t public_vms -T add x.x.x.212
pfctl -a wan_rules -f /tmp/rules.wan

# Sample rules.wan content
pass in quick on $wan proto tcp from any to x.x.x.211 port {22,80,443} flags S/SA keep state
pass in quick on $wan proto tcp from any to x.x.x.212 port {22,80,443} flags S/SA keep state
block in quick on $wan all

To maintain DHCP for public IPs while keeping pfSense firewall benefits:

  1. Disable pfSense DHCP server for public network
  2. Configure DHCP server on ESXi host or dedicated VM
  3. Example dhcpd.conf entries:
subnet x.x.x.0 netmask 255.255.255.0 {
  option routers x.x.x.1;
  option domain-name-servers 8.8.8.8, 8.8.4.4;
  
  host vm1 {
    hardware ethernet 00:50:56:01:23:45;
    fixed-address x.x.x.211;
  }
  host vm2 {
    hardware ethernet 00:50:56:67:89:ab;
    fixed-address x.x.x.212;
  }
}
  • Verify MAC addresses match between ESXi and DHCP configurations
  • Check vSwitch security policies (promiscuous mode, MAC changes)
  • Monitor pfSense state table for NAT issues
  • Test connectivity from both public and private VMs