Solved

ESXi 4 Networking Configuration - Need help

Posted on 2010-09-21
16
906 Views
Last Modified: 2012-08-13
We are trying to configure simple networking on a few ESXi servers we have. Currently ther are 2 nics in the server and I had configured the management network on one nic. Then when I configure the other for the VM Network switch I am only able to have one VM connecting at a time.

I would like help sort of starting from scratch building the network. We have a switch split into 2 VLAN's.. one for server traffic, one for the iSCSI SAN traffic. We have 3 new ESXi servers and only a couple of them will be using the SAN for storage. The third will just use local storage.

If someone can guide me through a basic network config and answer a few questions along the way it would be much appreciated. Thank you .
0
Comment
Question by:tobyhansen
  • 7
  • 4
  • 3
  • +2
16 Comments
 
LVL 8

Expert Comment

by:ragnarok89
ID: 33725329
Hi, we have 2 esx servers on our network, and there were no special considerations needed to set up networking... I assume you are giving them static IPs? Are both nics on the same subnet?

Dumb question, but do you perhaps have the same VM active on multiple machines?
0
 
LVL 5

Expert Comment

by:Marinertek
ID: 33725379
Your best bet is to use a virtual Distributed Switch across the hosts - here is a pdf on VMWare Networking Concepts that should help.
virtual-networking-concepts.pdf
0
 
LVL 10

Expert Comment

by:BloodRed
ID: 33725382
What is your vSwitch configuration?  Are the three NICs on the ESXi hosts connected to trunked ports on your switch or are they access ports configured for a specific vLAN?

You can either create on vSwitch and a port group for each vLAN, or multiple vSwitches with a single port group.  As for only a single VM being able to connect, that is strange.  Did you modify the default number of ports when you created the port group?

0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33725730
Lets just start with one ESXi server. We have a single switch, carved into 2 VLAN's. One for server and one for iSCSI. The iSCSI vlan is private and is not trunked anywhere. the other VLAN is just trunked into our core as is the rest of the VLAN's on our network.

The host has 2 NIC's. Both plugged into the server VLAN. For this Host we are using local storage and not going to plug into the iSCSI vlan. I have one NIC with the managment traffic port assigned. The other is just has a port group and VM Network. 56 ports assigned to the port group.

I can blow away the networking from scratch if someone would like to walk through the set up with me. I understand it is a simple configuration. it makes no sense this isn't working. I have one VM that can get out, the other cannot. All stupid IP related configurations have been doublechecked to make sure it isn't something dumb. We have played around with the network quite a bit so starting from scratch may not be a bad idea.

Thanks !
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726150
The real odd thing here is that one of the VM's can get out, the other can't. Same switch, same vSwitch, same port group. Changed IP's to make sure it wasn't a dup somewhere. Any help is appreciated. THanks.
0
 
LVL 10

Expert Comment

by:BloodRed
ID: 33726412
It sounds like you have it configured correctly, would you be able to take a screenshot of the networking configuration page?
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726505
I have been playing around with every combination I can think of. Still, just the one VM will communicate out. Before, I just had one vswitch and everything off that.
net.jpg
0
 
LVL 5

Expert Comment

by:Marinertek
ID: 33726592
Which adapter type are you using on the VM's (both that can and cannot connect)? If you're using the VMXNET type you'll need to have VMTools installed on the virtual machine.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Author Comment

by:tobyhansen
ID: 33726654
Marinertek, I am running VMXNET 3 on all guests. And they do have VMTools installed. WIth this however i am going to try removing them and trying the old E1000 type just as a test. This is a very odd problem to have.  -- thanks
0
 
LVL 10

Accepted Solution

by:
BloodRed earned 500 total points
ID: 33726695
You don't happen to have port security or anything else on the physical switch that would lock the port down to one MAC address, do you?

The ESX configuration looks correct, there is nothing in that setup that would prevent one VM from communicating.
0
 
LVL 5

Expert Comment

by:Marinertek
ID: 33726752
You could test pinging from one VM to another, then do some trace-routes off the working VM's and the non-working VM to the outside, that would give you a good start to possible blocks if the E1000 adapter doesn't fix the issue.
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726806
Guys, tried the E1000 adapter. Didn't help. Tried re-installing VM Tools. Nothing. The switch is one of the new Cisco Small Business Pro switches - ESW-540-48. I will do a little digging to see if anyone else has had an issue with ESX and this switch. As for the ESX config, yeah I can't think of anything else that would cause problems. Thanks for all the help guys.
0
 
LVL 10

Expert Comment

by:BloodRed
ID: 33726917
As Marinertek mentioned, can you ping from one VM to the other?  When you do that and both VMs are on the same ESX host and vSwitch, all network traffic is kept local.  If that doesn't work, the problem isn't with your vSwitch or physical switch, it is something internal to the VM...
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33727395
BloodRed... you da man. For some reason port sec got enabled on the switch. Disabled it and VIOLA ! Thank you !
0
 
LVL 1

Author Closing Comment

by:tobyhansen
ID: 33727440
Thank you ! I had read over the need to disable port security for ESX but never thought someone or something would have enabled it. I had to give all the credit to BloodRed as he nailed the solution. Both members were helpful though.
0
 
LVL 26

Expert Comment

by:lnkevin
ID: 33727606
Few things to look at:

- Subnet mask. You want to make sure the subnet mask on these vms are matched with your physical switch. (same class Ex: class A 255.0.0.0, B 255.255.0.0.....)
- Gateway. Make sure they are all pointing to the physical switch (Cisco)
- If they (vms) and ping each other, your vswitch is OK. The problem lies on your subnet and Gateway.

After you sort out your problem, for advance set up, you may want to create load balancing failover on your virtual network. Team the physical cards to eliminate single point of failure. Here is your guide.

http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

K
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

When we have a dead host and we lose all connections to the ESXi, and we need to find a way to move all VMs from that dead ESXi host.
Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now