tobyhansen
asked on
ESXi 4 Networking Configuration - Need help
We are trying to configure simple networking on a few ESXi servers we have. Currently ther are 2 nics in the server and I had configured the management network on one nic. Then when I configure the other for the VM Network switch I am only able to have one VM connecting at a time.
I would like help sort of starting from scratch building the network. We have a switch split into 2 VLAN's.. one for server traffic, one for the iSCSI SAN traffic. We have 3 new ESXi servers and only a couple of them will be using the SAN for storage. The third will just use local storage.
If someone can guide me through a basic network config and answer a few questions along the way it would be much appreciated. Thank you .
I would like help sort of starting from scratch building the network. We have a switch split into 2 VLAN's.. one for server traffic, one for the iSCSI SAN traffic. We have 3 new ESXi servers and only a couple of them will be using the SAN for storage. The third will just use local storage.
If someone can guide me through a basic network config and answer a few questions along the way it would be much appreciated. Thank you .
Your best bet is to use a virtual Distributed Switch across the hosts - here is a pdf on VMWare Networking Concepts that should help.
virtual-networking-concepts.pdf
virtual-networking-concepts.pdf
What is your vSwitch configuration? Are the three NICs on the ESXi hosts connected to trunked ports on your switch or are they access ports configured for a specific vLAN?
You can either create on vSwitch and a port group for each vLAN, or multiple vSwitches with a single port group. As for only a single VM being able to connect, that is strange. Did you modify the default number of ports when you created the port group?
You can either create on vSwitch and a port group for each vLAN, or multiple vSwitches with a single port group. As for only a single VM being able to connect, that is strange. Did you modify the default number of ports when you created the port group?
ASKER
Lets just start with one ESXi server. We have a single switch, carved into 2 VLAN's. One for server and one for iSCSI. The iSCSI vlan is private and is not trunked anywhere. the other VLAN is just trunked into our core as is the rest of the VLAN's on our network.
The host has 2 NIC's. Both plugged into the server VLAN. For this Host we are using local storage and not going to plug into the iSCSI vlan. I have one NIC with the managment traffic port assigned. The other is just has a port group and VM Network. 56 ports assigned to the port group.
I can blow away the networking from scratch if someone would like to walk through the set up with me. I understand it is a simple configuration. it makes no sense this isn't working. I have one VM that can get out, the other cannot. All stupid IP related configurations have been doublechecked to make sure it isn't something dumb. We have played around with the network quite a bit so starting from scratch may not be a bad idea.
Thanks !
The host has 2 NIC's. Both plugged into the server VLAN. For this Host we are using local storage and not going to plug into the iSCSI vlan. I have one NIC with the managment traffic port assigned. The other is just has a port group and VM Network. 56 ports assigned to the port group.
I can blow away the networking from scratch if someone would like to walk through the set up with me. I understand it is a simple configuration. it makes no sense this isn't working. I have one VM that can get out, the other cannot. All stupid IP related configurations have been doublechecked to make sure it isn't something dumb. We have played around with the network quite a bit so starting from scratch may not be a bad idea.
Thanks !
ASKER
The real odd thing here is that one of the VM's can get out, the other can't. Same switch, same vSwitch, same port group. Changed IP's to make sure it wasn't a dup somewhere. Any help is appreciated. THanks.
It sounds like you have it configured correctly, would you be able to take a screenshot of the networking configuration page?
ASKER
I have been playing around with every combination I can think of. Still, just the one VM will communicate out. Before, I just had one vswitch and everything off that.
net.jpg
net.jpg
Which adapter type are you using on the VM's (both that can and cannot connect)? If you're using the VMXNET type you'll need to have VMTools installed on the virtual machine.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805
ASKER
Marinertek, I am running VMXNET 3 on all guests. And they do have VMTools installed. WIth this however i am going to try removing them and trying the old E1000 type just as a test. This is a very odd problem to have. -- thanks
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
You could test pinging from one VM to another, then do some trace-routes off the working VM's and the non-working VM to the outside, that would give you a good start to possible blocks if the E1000 adapter doesn't fix the issue.
ASKER
Guys, tried the E1000 adapter. Didn't help. Tried re-installing VM Tools. Nothing. The switch is one of the new Cisco Small Business Pro switches - ESW-540-48. I will do a little digging to see if anyone else has had an issue with ESX and this switch. As for the ESX config, yeah I can't think of anything else that would cause problems. Thanks for all the help guys.
As Marinertek mentioned, can you ping from one VM to the other? When you do that and both VMs are on the same ESX host and vSwitch, all network traffic is kept local. If that doesn't work, the problem isn't with your vSwitch or physical switch, it is something internal to the VM...
ASKER
BloodRed... you da man. For some reason port sec got enabled on the switch. Disabled it and VIOLA ! Thank you !
ASKER
Thank you ! I had read over the need to disable port security for ESX but never thought someone or something would have enabled it. I had to give all the credit to BloodRed as he nailed the solution. Both members were helpful though.
Few things to look at:
- Subnet mask. You want to make sure the subnet mask on these vms are matched with your physical switch. (same class Ex: class A 255.0.0.0, B 255.255.0.0.....)
- Gateway. Make sure they are all pointing to the physical switch (Cisco)
- If they (vms) and ping each other, your vswitch is OK. The problem lies on your subnet and Gateway.
After you sort out your problem, for advance set up, you may want to create load balancing failover on your virtual network. Team the physical cards to eliminate single point of failure. Here is your guide.
http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf
K
- Subnet mask. You want to make sure the subnet mask on these vms are matched with your physical switch. (same class Ex: class A 255.0.0.0, B 255.255.0.0.....)
- Gateway. Make sure they are all pointing to the physical switch (Cisco)
- If they (vms) and ping each other, your vswitch is OK. The problem lies on your subnet and Gateway.
After you sort out your problem, for advance set up, you may want to create load balancing failover on your virtual network. Team the physical cards to eliminate single point of failure. Here is your guide.
http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf
K
Dumb question, but do you perhaps have the same VM active on multiple machines?