26 Jan How to Survive a Brutal VCF 9 DoD Deployment: ESX Hosts

If you have been following this series, you likely remember my first post, How to Survive a Brutal VCF 9 DoD Deployment: Lessons from the Field. In that introduction, we covered best practices and general considerations for working within the DoD or any high-security environment.

In this next installment, Trial and Error, I am diving straight into the specific mistakes we made and the technical roadblocks we hit. I will explain exactly how we navigated those challenges and what we learned during the process. My goal is to share these “lessons learned” so you can avoid the same pitfalls and keep your deployment on track.

The Host Networking “Gotcha”

This was not the first roadblock we encountered, but it is certainly one we wish we had identified sooner. Because this step happens so early in the process, I want to start here. When you install ESXi on your hosts, you must set them up the “right way” to ensure the management domain deployment succeeds.

Stick with Self-Signed Certificates

If you have experience installing ESXi, you know the basic setup is relatively simple. However, VCF 9 is incredibly particular about certificates. Here is a vital tip: leave the default self-signed certificates on your hosts during the deployment. Do not attempt to replace them with custom CA-signed certificates yet.

The Real Culprit: DNS Configuration

The actual “gotcha” lies in your host networking, specifically the DNS Configuration and Custom DNS Suffixes.

Traditionally, many admins enter the “short name” (for example: pl-w010-esx01) in the Hostname field and then add the domain to the Custom DNS Suffixes section. In VCF 9, this approach creates a conflict in how the system generates certificates. This mismatch leads to frustrating validation failures later in the process.

The Correct Configuration

To avoid this, follow these specific steps:

1. Leave the Custom DNS Suffixes section entirely blank.
2. Enter the FQDN (Fully Qualified Domain Name) directly into the Hostname field. Instead of just entering the short name, you should enter the complete name (e.g., pl-w010-esx01.yourdomain.mil) in that primary Hostname box. This ensures the host generates its internal certificates correctly and allows VCF to validate the host without any naming conflicts.

Five Things You Must Do on Each Host

While we are discussing host preparation, let’s cover the other essential settings you need. I think we can all agree that official technical documentation isn’t always the easiest to follow. You often start on one page, click a prerequisite link, and before you know it, you have nine tabs open and feel completely overwhelmed.

To save you the headache, I have consolidated the “must-have” configurations into one clear list.

1. Configure the network on each host.
   https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/deployment/deploying-a-new-vmware-cloud-foundation-or-vmware-vsphere-foundation-private-cloud-/preparing-your-environment/preparing-esx-hosts-for-vmware-cloud-foundation-or-vmware-vsphere-foundation/configure-the-network-on-vmware-cloud-foundation-hosts.html
   The DoD Reality Check: The official documentation lists setting the VLAN ID as optional. However, in our environment, the hosts remained completely silent until we explicitly tagged the Management VLAN. Don’t leave this to chance; set it during the initial setup.
2. Configure the Virtual Machine Network Port Group
   https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/deployment/deploying-a-new-vmware-cloud-foundation-or-vmware-vsphere-foundation-private-cloud-/preparing-your-environment/preparing-esx-hosts-for-vmware-cloud-foundation-or-vmware-vsphere-foundation/configure-the-virtual-machine-network-port-group-on-vmware-cloud-foundation-hosts.html
   After the initial DCUI setup, log into the ESXi host client (GUI) to adjust your port groups
   Key Security Tip: If you are in a DoD environment, do not join the Customer Experience Improvement Program (CEIP). Security protocols generally forbid this kind of outbound data sharing.
3. Set vSwitch MTU to 9000 (Jumbo Frames)
   If you are using Jumbo Frames—and for a high-performance VCF environment, you definitely should—you must update the MTU on the virtual switch. You can do this quickly while you are already logged into the host GUI:
   Navigate to Networking in the left-hand menu.
   Select the Virtual Switches tab.
   Select vSwitch0 and click Edit settings (or find it under the Actions menu).
   Change the MTU to 9000 and click Save.
4. Configure NTP
   https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/deployment/deploying-a-new-vmware-cloud-foundation-or-vmware-vsphere-foundation-private-cloud-/preparing-your-environment/preparing-esx-hosts-for-vmware-cloud-foundation-or-vmware-vsphere-foundation/configure-ntp-on-vmware-cloud-foundation-hosts.html
   Time synchronization is the backbone of VCF stability. Ensure every host points to your authorized internal NTP servers. If your clocks drift even slightly, the Bring-up process will likely fail during the security handshake phases.
5. Regenerate the Self-Signed Certificates
   https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/deployment/deploying-a-new-vmware-cloud-foundation-or-vmware-vsphere-foundation-private-cloud-/preparing-your-environment/preparing-esx-hosts-for-vmware-cloud-foundation-or-vmware-vsphere-foundation/regenerate-the-self-signed-certificate-on-esx-hosts.html
   This step is a non-negotiable requirement. Failing to do this almost guarantees validation errors during the deployment. Even though these are self-signed, they must be “fresh” and reflect the FQDN changes we discussed earlier.
   Steps:
   Log into the host via SSH (PuTTY) or use the Shell from the Direct Console User Interface (DCUI) by pressing Alt + F1 (you may need to enable ESXi Shell and/or SSH from Troubleshooting Options first)
   Run:
   

/sbin/generate-certificates

/sbin/generate-certificates

Reboot the host. Once the host comes back up, it is officially ready for the Management Domain deployment.

Preparing your ESX hosts correctly is the first major hurdle in a VCF 9 deployment. By focusing on these specific DNS and certificate requirements, you build a stable foundation that prevents the “Validation Failed” errors that plague so many secure environments.

It’s easy to get lost in the sea of tech docs, but hopefully, this consolidated list gives you the clarity you need to move forward with confidence.

Join the Conversation

I would love to hear from you. Have you encountered a specific host-side “gotcha” that didn’t make it into the official manual? Are you dealing with a unique hardware constraint in your own environment? Leave a comment below and let’s talk through the solutions together. Your experience might be exactly what another admin needs to get through their own “perfect storm.”

Continue the Journey

This post is part of a series dedicated to navigating the complexities of VMware Cloud Foundation 9.