We are running into a strange issue with setting up a private VLAN for Exchange. Running in a VMware ESXi 5.5 environment with a distributed switch, we have 3 Mailbox servers that each have a "public" virtual NIC on the LAN and a "private" virtual NIC, which is currently still on the same VLAN as the public, but with a "private" IP address.
I created another port group in the distributed switch with a separate VLAN ID (300). I did not change the private IP addresses, but as soon as I placed the private VM NICs into the separate port group, the 3 Mailbox servers' private IPs could no longer ping each other. And when I ran a tracert from any of those VMs to one of the other private IPs, the machines were trying to route through the public LAN interface. That's what is stumping me. The private IP addresses are on the same network segment, so why would Windows suddenly try to route traffic through the public interfaces? I looked at the routing table, and the routes looked correct. I've done this before on another ESXi cluster that doesn't have Enterprise Plus, hence no distributed switch, so I had to configure the port groups individually on each host, but it works fine there.