Intro
In the previous post in this series we created VLAN interfaces and networks on our pfSense virtual router appliance, created firewall rules to allow traffic, and then enabled VLAN trunking on the ports our NUCs were connected to on our physical switch. In this post we are going to created our nested ESXi hosts. Again, like I have said previously, this is one way to do it, there are many permutations. Goal here is to get up and running asap. Nothing stopping you implementing a different way now or later.
Complete list of blog post in this series…
Deploy a Nested ESXi Host
Creating nested ESXi hosts is actually quite an easy task, as its essentially just creating a VM! Plus, in the more recent vSphere releases, VMware have made it easier as even offering ESXi as an OS in the drop down even though nested ESXi is not officially supported. OK, lets get started…
Right click one of your physical ESXi host and click New Virtual Machine.
Select Create a new virtual machine
Give the VM a name and optionally a folder location.
Select a compute resource
Select a datastore
Select ESXi 6.7 and later for the drop down for compatibility
Select Other from the drop down for Guest OS Family and VMware ESXi 6.5 or later from the drop down for Guest OS Version. The warning can be ignored, its essentially saying nested ESXi is not officially supported.
For CPU select 8 from the drop down. Note, you can only select as many CPUs as you physically have in terms of logical processors on the physical ESXi host. Plus, we need 8 CPUs to run NSX-T Edges! Expand CPU and check/tick Hardware virtualization check box.
For Memory, enter 24GB. I personally don’t allocated all 32GB of my NUCs memory so that some is left for core VMs running eg VCSA, jumps, DNS, router etc etc. For New Hard Disk, enter 8GB and change Disk Provisioning to Thin Provision.
For New Network, change the port group to nested-trunk which we created earlier. By default, a VM only has one network adapter so we need to add 3 more for a total of 4. Click ADD NEW DEVICE and select Network Adapter. Again change the port group for each to nested-trunk.
Review the Ready to complete summary and click Finish
Install ESXi
Next we must install ESXi on the VM. In this example I installed ESXi 6.5 as at the time of building out my nested PKS NSX-T env, PKS didn’t support ESXi 6.7 which it does now. There are several ways to as which you can install ESXi such as from ISO on client device or from ISO on datastore. Below I use an ISO on my laptop.
Launch the remote console for the VM. Then click VMRC -> Manage -> Virtual Machine Settings. Click CD/DVD drive, then check the Connect at power on check box, then click Use ISO image file and then browse to where you have the ESXi ISO saved and click OK.
Click the green play button to start the VM. It will then start booting the ESXi installer from the ESXi ISO.
After a few moment the ESXi installer welcome screen will be presented
Hit enter and answer the onscreen prompts such as disk to install ESXi on, keyboard layout, and root password.
Once the install is complete, disconnect the ISO by clicking VMRC -> Removable Devices -> CD/DVD drive 1 -> Disconnect. Hit enter to reboot the VM.
After a few moments the nested ESXi host will have booted and you will see the familiar DCUI screen.
Just like the physical ESXi hosts, we need to configure the management network. Hit F2 and enter the root password set during the install. Navigate to Configure Management Network and then VLAN. Enter 10 for the VLAN ID and hit enter. Remember it was VLAN 10 we configured on the pfSense router for the management network in the previous post.
Navigate to IPv4 Configuration and select the Set static IPv4 address and network configuration option. Enter an IP address from our 10.0.10.0/24 network we created in the previous post, followed by a subnet mask, and finally the default gateway we also created previously. Hit Enter.
Navigate to DNS Configuration and enter the IP address of the DNS server we created previously followed by a hostname for this nested ESXi host. Hit Enter. Don’t forget to create an entry in the DNS for this host!
To save all these settings hit ESC and you will then be prompted to apply the changes and restart the management network to which you respond yes.
Now our nested ESXi host has its management network configured. Wuhoo!
Now the moment of truth, can we actually reach it! Eeeeek! On the jump we created previously, we configured it to have two nics, one in the 192.168.0.0/24 network (VLAN 1) and the other in the 10.0.10.0/24 network (VLAN 10). On the jump host, open a browser to the management IP of the nested ESXi host, in this case, 10.0.10.11….
…and it works. Wuhoo. OK, celebration over, we have to do all this 2 more times for a total of 3 nested ESXi hosts. Naturally don’t use the same names of IPs, increment by one. As I have said previously, this can be known as the “hard way”. All these steps can be automated but I believe if you understand the “hard way” then it makes the “easy way” easy! Plus, if it breaks, you will know how to fix it.
That complete this blog post. Phew, that was a lot of screenshots!
