Solved

ESXi 4 Networking Configuration - Need help

Posted on 2010-09-21
16
910 Views
Last Modified: 2012-08-13
We are trying to configure simple networking on a few ESXi servers we have. Currently ther are 2 nics in the server and I had configured the management network on one nic. Then when I configure the other for the VM Network switch I am only able to have one VM connecting at a time.

I would like help sort of starting from scratch building the network. We have a switch split into 2 VLAN's.. one for server traffic, one for the iSCSI SAN traffic. We have 3 new ESXi servers and only a couple of them will be using the SAN for storage. The third will just use local storage.

If someone can guide me through a basic network config and answer a few questions along the way it would be much appreciated. Thank you .
0
Comment
Question by:tobyhansen
  • 7
  • 4
  • 3
  • +2
16 Comments
 
LVL 8

Expert Comment

by:ragnarok89
ID: 33725329
Hi, we have 2 esx servers on our network, and there were no special considerations needed to set up networking... I assume you are giving them static IPs? Are both nics on the same subnet?

Dumb question, but do you perhaps have the same VM active on multiple machines?
0
 
LVL 5

Expert Comment

by:Marinertek
ID: 33725379
Your best bet is to use a virtual Distributed Switch across the hosts - here is a pdf on VMWare Networking Concepts that should help.
virtual-networking-concepts.pdf
0
 
LVL 10

Expert Comment

by:BloodRed
ID: 33725382
What is your vSwitch configuration?  Are the three NICs on the ESXi hosts connected to trunked ports on your switch or are they access ports configured for a specific vLAN?

You can either create on vSwitch and a port group for each vLAN, or multiple vSwitches with a single port group.  As for only a single VM being able to connect, that is strange.  Did you modify the default number of ports when you created the port group?

0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 1

Author Comment

by:tobyhansen
ID: 33725730
Lets just start with one ESXi server. We have a single switch, carved into 2 VLAN's. One for server and one for iSCSI. The iSCSI vlan is private and is not trunked anywhere. the other VLAN is just trunked into our core as is the rest of the VLAN's on our network.

The host has 2 NIC's. Both plugged into the server VLAN. For this Host we are using local storage and not going to plug into the iSCSI vlan. I have one NIC with the managment traffic port assigned. The other is just has a port group and VM Network. 56 ports assigned to the port group.

I can blow away the networking from scratch if someone would like to walk through the set up with me. I understand it is a simple configuration. it makes no sense this isn't working. I have one VM that can get out, the other cannot. All stupid IP related configurations have been doublechecked to make sure it isn't something dumb. We have played around with the network quite a bit so starting from scratch may not be a bad idea.

Thanks !
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726150
The real odd thing here is that one of the VM's can get out, the other can't. Same switch, same vSwitch, same port group. Changed IP's to make sure it wasn't a dup somewhere. Any help is appreciated. THanks.
0
 
LVL 10

Expert Comment

by:BloodRed
ID: 33726412
It sounds like you have it configured correctly, would you be able to take a screenshot of the networking configuration page?
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726505
I have been playing around with every combination I can think of. Still, just the one VM will communicate out. Before, I just had one vswitch and everything off that.
net.jpg
0
 
LVL 5

Expert Comment

by:Marinertek
ID: 33726592
Which adapter type are you using on the VM's (both that can and cannot connect)? If you're using the VMXNET type you'll need to have VMTools installed on the virtual machine.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726654
Marinertek, I am running VMXNET 3 on all guests. And they do have VMTools installed. WIth this however i am going to try removing them and trying the old E1000 type just as a test. This is a very odd problem to have.  -- thanks
0
 
LVL 10

Accepted Solution

by:
BloodRed earned 500 total points
ID: 33726695
You don't happen to have port security or anything else on the physical switch that would lock the port down to one MAC address, do you?

The ESX configuration looks correct, there is nothing in that setup that would prevent one VM from communicating.
0
 
LVL 5

Expert Comment

by:Marinertek
ID: 33726752
You could test pinging from one VM to another, then do some trace-routes off the working VM's and the non-working VM to the outside, that would give you a good start to possible blocks if the E1000 adapter doesn't fix the issue.
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33726806
Guys, tried the E1000 adapter. Didn't help. Tried re-installing VM Tools. Nothing. The switch is one of the new Cisco Small Business Pro switches - ESW-540-48. I will do a little digging to see if anyone else has had an issue with ESX and this switch. As for the ESX config, yeah I can't think of anything else that would cause problems. Thanks for all the help guys.
0
 
LVL 10

Expert Comment

by:BloodRed
ID: 33726917
As Marinertek mentioned, can you ping from one VM to the other?  When you do that and both VMs are on the same ESX host and vSwitch, all network traffic is kept local.  If that doesn't work, the problem isn't with your vSwitch or physical switch, it is something internal to the VM...
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33727395
BloodRed... you da man. For some reason port sec got enabled on the switch. Disabled it and VIOLA ! Thank you !
0
 
LVL 1

Author Closing Comment

by:tobyhansen
ID: 33727440
Thank you ! I had read over the need to disable port security for ESX but never thought someone or something would have enabled it. I had to give all the credit to BloodRed as he nailed the solution. Both members were helpful though.
0
 
LVL 26

Expert Comment

by:lnkevin
ID: 33727606
Few things to look at:

- Subnet mask. You want to make sure the subnet mask on these vms are matched with your physical switch. (same class Ex: class A 255.0.0.0, B 255.255.0.0.....)
- Gateway. Make sure they are all pointing to the physical switch (Cisco)
- If they (vms) and ping each other, your vswitch is OK. The problem lies on your subnet and Gateway.

After you sort out your problem, for advance set up, you may want to create load balancing failover on your virtual network. Team the physical cards to eliminate single point of failure. Here is your guide.

http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

K
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this step by step tutorial with screenshots, we will show you HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 6.5 (ESXi 6.5). This is important if you need to enable SSH remote access for additional troubleshooting of the ESXi hos…
In this article, I will show you HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, the Windows OS we will install is Windows Server 2016.
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question