No traffic over trunked ports on C2960 ; non-trunked access ports Ok on ESXi hosts

Refer to attached screen.

We have a pair of C2960-48TS L2 stacked switches
that connects to our ESXi hosts.  The management
& vMotion ports on the ESXi hosts are connected
to access (ie non-trunked) ports on these switches
& they're Ok (ie management ports could be pinged
from elsewhere).

However, for the Prod VLANs, we connect up to
trunked ports on these switches but we could only
see VLAN 1 in the dvSwitch (see attached).

There are VMs in each of the four VLans permitted
in the trunked ports but all the VMs can't even
ping their respective VLANs' gateways.

The network guys told me I cannot set
    "switchport trunk encapsulation dot1q" on the
trunked interfaces on the switches as this command
is by default (or implicitly) enabled on this L2 C2960
switch.

The vmnics that connect to these switches' trunked
ports in the vCenter showed they're down.   In another
site's setup (same model of switch & ESXi hosts) where
all the ports on the switches are set to 'non-trunk' (ie
access ports), we can see all the four VLANs in vCenter
(refer to attached) though each vmnic only showed one
VLAN (ie certain ports showed VLAN 139, certain other
ports showed 423, yet other ports showed VLan 452).

Why is it all my VMs can't ping their respective VLANs'
gateways?  What did I miss?  Is there such thing in
the ESXi hosts that we set trunked ports or something
needs to be done in the IBM x3850 X5's BIOS setting?

I heard that in another project, they used C3750 stacked
switches which allows "switchport trunk encapsulation
dot1q" & they did not face this issue.

"Show run" outputs of the trunked ports on the switches:

interface GigabitEthernet1/0/3
 switchport trunk allowed vlan 139,423,452,454
 switchport mode trunk
 switchport nonegotiate
 speed 1000
 duplex full
 flowcontrol receive desired
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
end

interface GigabitEthernet1/0/4
 switchport trunk allowed vlan 139,423,452,454
 switchport mode trunk
 switchport nonegotiate
 speed 1000
 duplex full
 flowcontrol receive desired
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
end

interface GigabitEthernet2/0/1
 switchport trunk allowed vlan 139,423,452,454
 switchport mode trunk
 switchport nonegotiate
 speed 1000
 duplex full
 flowcontrol receive desired
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
end

interface GigabitEthernet2/0/2
 switchport trunk allowed vlan 139,423,452,454
 switchport mode trunk
 switchport nonegotiate
 speed 1000
 duplex full
 flowcontrol receive desired
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
end

. . .
dvSwitchVlan1.jpg
sunhuxAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sunhuxAuthor Commented:
"Line protocol" is Up on all the trunked ports but
simply traffic can't get through.  I still want trunk
ports & not access ports as need to allow multiple
VLANs on each interface
0
sunhuxAuthor Commented:
If trunking is removed, it would work but a non-trunked
(ie access) ports only allow one VLAN, right?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
does not look like traffic is tagged on the VLANs

VLAN1 is the default.

You are correct, access VLAN is single VLAN on an access port.

Multiple VLANs, are tagged and trunked on a port.

we normally use

switchport trunk encapsulation dot1q

on all our Cisco trunk ports.
0
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

sunhuxAuthor Commented:
So how exactly do I solve this issue?

Is this a Cisco C2960 issue that needs to be rectified
or it's something I can do in vCenter?

>does not look like traffic is tagged on the VLANs
how do we tag the traffic then?
0
sunhuxAuthor Commented:
We've verified that on C2960, the command below can't be accepted:
  switchport trunk encapsulation dot1q
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
So how does your switch Tag VLAN traffic?
0
sunhuxAuthor Commented:
The network chaps who manage the C2960 layer 2 switches told
me for this model of switch, all trunked ports would have
"switchport trunk encapsulation dot1q" implicitly enabled by
default though the switch disallows this command from
being entered
0
sunhuxAuthor Commented:
So by this implicit  'encap dot1q' does it mean VLan tagging
is automatically done?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
That does seem to suggest, it's on by default.

Do you use these VLANs on any other equipment or server, to test VLANs are accessible?

I would raise a Request for Assistance, and try and get a response from a Cisco Engineer familiar with the switch hardware you have.
0
sunhuxAuthor Commented:
Those IBM chaps who support the network switches are CCNPs;
they insisted this connectivity issue lies with ESXi hosts end but
the ESXi hosts' chaps are VCPs
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I find it odd, that vSphere states it can only see VLAN 1, which is a default VLAN on most network kit.

Personally, I never know any Cisco kit, we've come across that does not accept the switchport trunk encapsulation dot1q configuration command. That's it not to say that you Cisco guys are right or wrong, but I would test with other server/devices to doublecheck.

I would raise a Request for Assistance, and try and get a response from a Cisco Engineer (CCIE) familiar with the switch hardware you have, or other eyes on this!

You could always escalate to VMware Support.
0
TimotiStDatacenter TechnicianCommented:
The 2960 is a basic switch, it can't do ISL trunking like the big or old boxes, so it really defaults to dot1q.

Possibly a spanning tree issue, if the ESX is bridging to NICs?
Or (unlikely) an ESX driver issue and it can't tag on the NICs of those servers?

Tamas
0
sunhuxAuthor Commented:
So how do I bridge the ESXi's NIC to the switches?

As last resort, may need to set all ports to access (ie non-trunk)
ports & designate certain VLAN to certain port, say we have
8 switch/NIC ports, each VLAN gets 2 ports
0
btassureCommented:
Can you post the properties of the dvSwitch as well please? The Cisco settings look OK, and the reason you can't enter the dot1q stuff is that there is no support for ISL on this model so there is way to change the command anyway.
0
Netman66Commented:
First thing I would do is set the Duplex and Speed to Auto/Auto on the switch.  Some NICs hate being locked and should negotiate at GB on their own.

Secondly, why are you teaming all those NICs on a single vSwitch with all those vlans?

We created a vSwitch per vlan and left the switchports as access ports in the specific vlan they serve.  The switch or first layer 3 device inline should handle intra-vlan routing for you on it's own.  Besides, trunking all those ports into that team has it's own set of problems as the ESXi stack isn't nearly as "smart" as your switch.
0
btassureCommented:
I would imagine it was done for the same reasons we do it - if you have 4 NICs for example and 4 VLANs, and 3 of them are mostly idle you will have 3 NICs doing nothing most of the time while you have 1 of them serving your active vSwitch at 1Gbps while mine will have 4. Yes there are some potential contention issues and ESXi isn't as clever as the physical switches but you can let the Cisco switches do the hard work.

I would say either approach has merit though!
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sunhuxAuthor Commented:
> why are you teaming all those NICs on a single vSwitch with all those vlans?
thanks btassure for replying to Netman66

> Duplex and Speed to Auto/Auto
The IBM CCNPs told me they've tried both auto  as well as fixed settings (ie
Duplex Full & Speed 1000M).  Question is why are the non-trunk ports Ok
with these fixed settings?

Ok, attached are the various Properties screen shots of the dvSwitch
as requested.  Let me know any other screens/info that's needed as
customer is getting impatient.  There are 3 ESXi hosts
Sproblem-dvSwitch.jpg
Sproblem-dvSwitchProperts1.jpg
Sproblem-dvSwitchProperts2.jpg
Sproblem-dvSwitch-NetwAdp.jpg
Sproblem-dvSwitch-PteVlan.jpg
Sproblem-dvSwitch-Portmirrg.jpg
0
sunhuxAuthor Commented:
Any other ideas from the experts?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware

From novice to tech pro — start learning today.