random neuron misfires

One of my machines in my production homelab is an ESXi server, a long-toothed upgrade from the 5.x days.

I keep a lot of legacy VMs and copies of every version of Ubuntu, Fedora, Slackware, Debian, CentOS and hundreds of other VMs on it. It’s invaluable to be able to spin up a test machine on any OS, any capacity, within seconds.

Recently, the need to ramp up fast on Proxmox has come to the front of my priority list for work and specific customer needs. I don’t have spare, baremetal hardware to install Proxmox natively, so I have to spin it up under my existing VMware environment as a guest.

The problem here, is that running one hypervisor under another hypervisor as a guest, requires some specific preparations, so that the networking of the nested guest, will have its packets correctly and cleanly routed through the parent host’s physical network interfaces.

Read on for how to configure this in your own environment!

VMware ESXi

In VMware ESXi, there are a few settings that you need to adjust, to enable “Promiscuous Mode”, “Forged Transmits” and “MAC Changes”. These are found under the “VM Network” section of your ESXi web-ui:

Once you’ve made these changes, you need to restart your VMware host in order to enable them for newly-created VMs under that host.

VMware Workstation

If you’re running VMware Workstation instead of ESXi, you need to make sure your ‘vmnet’ devices in /dev/ have the correct permissions to permit enabling promiscuous mode. You can do that with a quick chmod 0777 /dev/vmnet* or you can adjust the VMware init script that creates these nodes. Normally these would be adjusted in ‘udev’ rules, but those rules are run before the VMware startup, so changes are overwritten by VMware’s own automation.

In /etc/init.d/vmware, make the following adjustment:

vmwareStartVmnet() {
   vmwareLoadModule $vnet
   "$BINDIR"/vmware-networks --start >> $VNETLIB_LOG 2>&1
   chmod 666 /dev/vmnet* # Add this line
}

Now that you have your host hypervisor configured to support nested guest hypervisors, let’s proceed with the Proxmox installation.

Download the most-recent Proxmox ISO image and create a new VM in your VMware environment (ESXi or Workstation). Make sure to give your newly created VM enough resources to be able to launch its own VMs. I created a VM with 32GB RAM and 2TB of storage, configured as a ZFS RAIDZ-3 array (5 x 400GB disks). That configuration looks like this:

Read the rest of this entry »