Need help configuring Catalyst 6500 and ESX 4.0 for VLAN Trunking


I am trying to configure a trunk so that I can use multiple subnets on my ESX Host.  I am having some issues getting it to work and the biggest issue is that while configurring it, I don't really know how to test my setup to try and find the problem.  The basic instructions I followed from here:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004074

are below ....    (I don't need crossover cable like switch to switch trunk for this do I?)


The following sample is a supported Cisco Trunk Port configuration:

    interface GigabitEthernet2/3
    switchport                                      
    switchport trunk encapsulation dot1q  
    switchport trunk allowed vlan all
    switchport mode trunk                      
    switchport nonegotiate                    
    spanning-tree portfast trunk            

To assign a VLAN to a port group,  there has to be a corresponding VLAN interface for each VLAN on a physical switch with a designated IP range.

    For example:

    interface Vlan3
    ip address 192.168.3.1 255.255.255.0


To configure VLAN on the portgroup within the Virtual Infrastructure Client:

   1. Highlight the ESX host.
   2. Click the Configuration tab.
   3. Click the Networking link.
   4. Click Properties.
   5. Highlight the virtual switch in the Ports tab and click Edit.
   6. Click the General tab.
   7. Assign a VLAN number in VLAN ID (optional).
   8. Click the NIC Teaming tab.
   9. From the Load Balancing dropdown, choose Route based on originating virtual port ID.
  10. Verify that there is at least one network adapter listed underActive Adapters.

This doesn't work becasue the trunk never gets established ...
  11. Verify VST configuration by utilizing the ping command to confirm connection between ESX host and gateway interfaces and other host on the same VLAN.



basically when done, I just lose connectivity and not sure how to troubleshoot.

Thanks
DanRaposoAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

65tdRetiredCommented:
Has a native trunk being configured?
Are the vlan numbers being entered in the trunk config?

If trunking is working should be able to switch the VM between networks and DHCP or assign an IP address on the subnet.
Also see this vmware doc on 802.1q trunking:
http://www.vmware.com/pdf/esx3_vlan_wp.pdf
0
Paul SolovyovskySenior IT AdvisorCommented:
The first link is actuall an example of a Cisco 6500
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

DanRaposoAuthor Commented:
Paul,

The first link is an example of link aggregation.   What I want to do is trunk multiple VLANs over the same switch Port / through the same NIC to an ESXi server.   I have read most of those documents, but I must be missing something.  
0
Paul SolovyovskySenior IT AdvisorCommented:
It should be the same.  Create a trunk port on the switch and connect to a single nic.  Create a vswitch and add port groups (each port group is a vlan, label accordingly).  Set the vswitch to IP Hash and you should be set
0
DanRaposoAuthor Commented:
Sorry for the delay here ...   I have setup of the port to trunk VLANs and created the vSwitch.   I have also added a port group, but no servers (guests) are being assigned to that group.  I don't know what IP hash is.

0
Paul SolovyovskySenior IT AdvisorCommented:
load balancing config to 'route based on ip hash' on the vswitch
0
DanRaposoAuthor Commented:
I have moved this test to another switch and host so that I can be more proactive in testing.   Current setup is a Catalyst 4507e connecting to a macbook pro (192.168.0.xxx) network (VLAN 192)  with vm Fusion installed.
Running inside a vm is windows 2003 on VLAN 100 (192.168.100.xxx)

This config should be very similar to what i was trying to do to the esx host and should help me over the hurdle.

The Mac is connected to port 1/15 on the Cat4507  Here is config info


Name: Gi1/15
Switchport: Enabled
Administrative Mode: trunk
Operational Mode: trunk
Administrative Trunking Encapsulation: dot1q
Operational Trunking Encapsulation: dot1q
Negotiation of Trunking: On
Access Mode VLAN: 1 (default)
Trunking Native Mode VLAN: 1 (default)
Administrative Native VLAN tagging: enabled
Voice VLAN: none
Administrative private-vlan host-association: none
Administrative private-vlan mapping: none
Administrative private-vlan trunk native VLAN: none
Administrative private-vlan trunk Native VLAN tagging: enabled
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk associations: none
Administrative private-vlan trunk mappings: none
Operational private-vlan: none
Trunking VLANs Enabled: ALL
Pruning VLANs Enabled: 2-1001
Capture Mode Disabled
Capture VLANs Allowed: ALL

Unknown unicast blocked: disabled
Unknown multicast blocked: disabled
Appliance trust: none



interface GigabitEthernet1/15
 description test_port_dan
 switchport mode trunk
 spanning-tree portfast disable


Connected via straight through Cat 5 cable ....

Any glaring setup issues here?

Thanks
0
DanRaposoAuthor Commented:
I disabled VLAN pruning in hopes that was my issue but no luck

Pruning VLANs Enabled: NONE
0
DanRaposoAuthor Commented:
Results of sh spanning-tree makes me believe that the VLANS are enabled on the port, but yet no DHCP address assignment is happening nor can static ips communictae to the default gateway.

sh ip route results are at bottom of post.  192.168.0 and 192.168.100 are the subnets being tested.

Cat4507e# sh spanning-tree

VLAN0001
  Spanning tree enabled protocol ieee
  Root ID    Priority    24577
             Address     0004.4d7f.9f00
             Cost        4
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32769  (priority 32768 sys-id-ext 1)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0002
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec1
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32770  (priority 32768 sys-id-ext 2)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0003
  Spanning tree enabled protocol ieee
  Root ID    Priority    32771
             Address     0004.4d7f.9f00
             Cost        4
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32771  (priority 32768 sys-id-ext 3)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0005
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec2
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32773  (priority 32768 sys-id-ext 5)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0033
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec4
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32801  (priority 32768 sys-id-ext 33)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0037
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec5
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32805  (priority 32768 sys-id-ext 37)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0100
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec6
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32868  (priority 32768 sys-id-ext 100)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0101
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec7
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32869  (priority 32768 sys-id-ext 101)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0105
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec8
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32873  (priority 32768 sys-id-ext 105)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0126
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec9
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32894  (priority 32768 sys-id-ext 126)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0147
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cec3
             Cost        80
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32915  (priority 32768 sys-id-ext 147)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p


VLAN0192
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0002.7e66.cecb
             Cost        23
             Port        16 (GigabitEthernet1/16)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32960  (priority 32768 sys-id-ext 192)
             Address     0022.55d7.3f40
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/15              Desg FWD 4         128.15   P2p
Gi1/16              Root FWD 4         128.16   P2p
Fa2/48              Desg FWD 19        128.112  P2p Edge



RESULTS of sh ip route


S    192.168.10.0/24 [1/0] via 192.168.6.254
S    192.168.200.0/24 [1/0] via 192.168.0.15
S    192.168.157.0/24 [1/0] via 192.168.0.250
     10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
S       10.9.0.0/16 [1/0] via 192.168.0.2
S       10.9.10.0/24 [1/0] via 192.168.0.2
S       10.27.0.0/16 [1/0] via 192.168.0.2
S       10.77.0.0/16 [1/0] via 192.168.0.2
C    192.168.6.0/24 is directly connected, Vlan1
S    192.168.0.0/24 [1/0] via 192.168.6.1
S    192.168.100.0/24 [1/0] via 192.168.6.1
S*   0.0.0.0/0 [1/0] via 192.168.6.254




0
DanRaposoAuthor Commented:
Show I cancel this ?   I still don;t have it working, but it could be environment (NIC card problmes??)  and not that these answers are not right or helpful.
0
bgoeringCommented:
Fusion is going to be a bit different than ESX - you will have to decode the VLANs at the MAC host level and assign them to the vSwitches.

For your ESX if I understand you right you have a single physical NIC configured into a vSwitch instead of multple - load balancing and such doesn't apply with a single NIC. What you posted about the 6509 looks pretty much like what I use. On the ESX side configure a single vSwitch and associate the vmnic to it. Then create seperate port groups for each VLAN and assign the VLAN number to the port group. Leave your first port group blank where it asks for VLAN number (optional) and that will be associated with your native, untagged VLAN. It will look something like this:

6500 <---> vmnicX ---> vSwitch
                                         portgroup "Name1" VLAN none
                                         portgroup "Name100" VLAN 100
                                         portgroup "Name110" VLAN 110

Hope this helps, If you need to get into NIC teaming let me know and we can go over that
0
Luciano PatrãoICT Senior Infraestructure  Engineer  Commented:
Hi

If I understand your question correctly, for using VLAN(using also on the physical Switch port level)  in VMware with no other special configurations, you only need to create the portgroup with the VLAN ID that you want to use and also with the physical adapter port(vmnic) for that portgroup. Then connection this physical adapter port to the physical Switch port and configure on the physical Switch port also the same VLAN ID

This is what we do for a simple network VLANs network

Hope this can help you

Jail
0
Luciano PatrãoICT Senior Infraestructure  Engineer  Commented:
Hi

Just to add the configuration on the ESX host side

I use post install scripts to create the portgroups and team etc. I will add an example

lets create an example for vSwitch2

Create vSwitch
esxcfg-vswitch -a vSwitch2

Add/link the vmnics to vSwitch
esxcfg-vswitch -L vmnic1 vSwitch2
esxcfg-vswitch -L vmnic2 vSwitch2

Create teaming and load balance for this vSwitch
vmware-vim-cmd hostsvc/net/vswitch_setpolicy --nicorderpolicy-active=vmnic1,vmnic2 --nicteaming-policy=loadbalance_ip vSwitch2
vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-promisc=true vSwitch2

Enable Cisco Discovery
esxcfg-vswitch -B both vSwitch2

Create 2 portgroups with different VLANs as an example
esxcfg-vswitch -A "PortGroup01" vSwitch2
esxcfg-vswitch -p "PortGroup01" -v 64 vSwitch2

esxcfg-vswitch -A "PortGroup02" vSwitch2
esxcfg-vswitch -p "PortGroup02" -v 84 vSwitch2

service mgmt-vmware restart

Hope this can help you

Jail
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
DanRaposoAuthor Commented:
This is good info and may not be exactly what I am looking for but shuold help most people stumbling accross this link.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.