Solved

esxi 6 vlan networking not working on virtualized vmware workstation 12 hosted on ubuntu lts 16.04.1

Posted on 2016-07-28
2
85 Views
Last Modified: 2016-07-28
guest can ping the esxi server but no further. other systems in the same subnets cannot be reached. i have setup 3 vlans. this is the management vlan. the esxi server gets its ip via dhcp from bwell. so this is working. esxi server can ping everything in all vlans but client is not able to communicate. i tried tagging on portgroup and no tagging. with no tagging on the portgroup i can ping the esx server and with tagging i cannot ping.
 
this is the first problem. the next would be that none of the other networks can communicate with the outside world.
 
root@bwell:~# arping -I ens6 192.168.2.100
ARPING 192.168.2.100 from 192.168.178.47 ens6
Unicast reply from 192.168.2.100 [2C:3E:CF:FF:3A:2D]  1.604ms
Unicast reply from 192.168.2.100 [2C:3E:CF:FF:3A:2D]  1.595ms
Unicast reply from 192.168.2.100 [2C:3E:CF:FF:3A:2D]  1.576ms
Unicast reply from 192.168.2.100 [2C:3E:CF:FF:3A:2D]  1.524ms
^CSent 4 probes (1 broadcast(s))
Received 4 response(s)
 
root@bwell:~# ping 192.168.2.100
PING 192.168.2.100 (192.168.2.100) 56(84) bytes of data.
^C
--- 192.168.2.100 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3022ms
 
[root@esx1:~] vmkping 192.168.2.1
PING 192.168.2.1 (192.168.2.1): 56 data bytes
64 bytes from 192.168.2.1: icmp_seq=0 ttl=64 time=0.628 ms
 
--- 192.168.2.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.628/0.628/0.628 ms
 
[root@esx1:~] esxcfg-vmknic -l
Interface  Port Group/DVPort/Opaque Network        IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type                NetStack          
vmk0       Management Network                      IPv4      192.168.2.3                             255.255.255.0   192.168.2.255   00:0c:29:3f:02:e8 1500    65535     true    DHCP                defaultTcpipStack  
vmk0       Management Network                      IPv6      fe80::20c:29ff:fe3f:2e8                 64                              00:0c:29:3f:02:e8 1500    65535     true    STATIC, PREFERRED   defaultTcpipStack  
vmk1       ProdNet                                 IPv4      192.168.1.201                           255.255.255.0   192.168.1.255   00:50:56:63:c2:10 1500    65535     true    STATIC              prodtcpip          
vmk1       ProdNet                                 IPv6      fe80::250:56ff:fe63:c210                64                              00:50:56:63:c2:10 1500    65535     true    STATIC, PREFERRED   prodtcpip          
vmk2       BackupNet                               IPv4      192.168.3.201                           255.255.255.0   192.168.3.255   00:50:56:6e:97:b0 1500    65535     true    STATIC              backuptcpip        
vmk2       BackupNet                               IPv6      fe80::250:56ff:fe6e:97b0                64                              00:50:56:6e:97:b0 1500    65535     true    STATIC, PREFERRED   backuptcpip        
vmk3       Internet                                IPv4      192.168.178.203                         255.255.255.0   192.168.178.255 00:50:56:6b:81:c6 1500    65535     true    STATIC              intertcpip        
vmk3       Internet                                IPv6      fe80::250:56ff:fe6b:81c6                64                              00:50:56:6b:81:c6 1500    65535     true    STATIC, PREFERRED   intertcpip        
vmk3       Internet                                IPv6      2a02:810d:8ac0:26c4:250:56ff:fe6b:81c6  64                              00:50:56:6b:81:c6 1500    65535     true    AUTOCONF, PREFERRED intertcpip        
 
[root@esx1:~] esxcli network ip route ipv4 list
Network      Netmask        Gateway      Interface  Source
-----------  -------------  -----------  ---------  ------
default      0.0.0.0        192.168.2.1  vmk0       DHCP
192.168.2.0  255.255.255.0  0.0.0.0      vmk0       MANUAL
 
this is the routing table from the switch
 
switchff3a2d#show ip route
Maximum Parallel Paths: 1 (1 after reset)
IP Forwarding: enabled
Codes: > - best, C - connected, S - static
 
 
S   0.0.0.0/0 [1/1] via 192.168.178.1, 43:47:32, vlan 1                  
C   192.168.1.0/24 is directly connected, vlan 100                        
C   192.168.2.0/24 is directly connected, vlan 200                        
C   192.168.3.0/24 is directly connected, vlan 300                        
C   192.168.178.0/24 is directly connected, vlan 1                        
 
switchff3a2d#show ip route address 192.168.2.3
Maximum Parallel Paths: 1 (1 after reset)
IP Forwarding: enabled
Codes: > - best, C - connected, S - static
 
 
S   0.0.0.0/0 [1/1] via 192.168.178.1, 44:44:31, vlan 1                  
C   192.168.2.0/24 is directly connected, vlan 200                        
 
switchff3a2d#show ip route address 192.168.2.100
Maximum Parallel Paths: 1 (1 after reset)
IP Forwarding: enabled
Codes: > - best, C - connected, S - static
 
 
S   0.0.0.0/0 [1/1] via 192.168.178.1, 44:44:33, vlan 1                  
C   192.168.2.0/24 is directly connected, vlan 200                        
 
switchff3a2d#show vlan          
Created by: D-Default, S-Static, G-GVRP, R-Radius Assigned VLAN, V-Voice VLAN
 
Vlan        Name            Tagged Ports       UnTagged Ports       Created by  
---- ----------------- ------------------ ------------------ ----------------
1            1                             fa1/2/1-48,                V         
                                           gi1/1/1-7,                          
                                           gi1/1/9-52,                         
                                           gi1/2/1-4,                          
                                           fa2/2/1-48,                         
                                           gi2/1/1-52,                         
                                           gi2/2/1-4,                          
                                           fa3/2/1-48,                         
                                           gi3/1/1-52,                         
                                           gi3/2/1-4,                          
                                           fa4/2/1-48,                         
                                           gi4/1/1-52,                         
                                           gi4/2/1-4,                          
                                           fa5/2/1-48,                         
                                           gi5/1/1-52,                         
                                           gi5/2/1-4,                          
                                           fa6/2/1-48,                         
                                           gi6/1/1-52,                         
                                           gi6/2/1-4,                          
                                           fa7/2/1-48,                         
                                           gi7/1/1-52,                         
                                           gi7/2/1-4,                          
                                           fa8/2/1-48,                         
                                           gi8/1/1-52,                         
                                           gi8/2/1-4,Po1-32                    
100         prod         gi1/1/2-7,gi1/1/13       gi1/1/1               S         
200         mgmt         gi1/1/2-7,gi1/1/13  gi1/1/1,gi1/1/8           S         
300        backup        gi1/1/2-7,gi1/1/13       gi1/1/1               S         
 
somebody please help. i really dont know anymore what to do...
 
i wanted to add that the host for the esxi server is vmware workstation pro 12 and this is running on ubuntu 16.04.1
 
 
 
                 
root@bwell:~# ifconfig
enp10s0   Link encap:Ethernet  HWaddr 68:05:ca:42:ff:b9
          inet addr:192.168.4.1  Bcast:192.168.4.255  Mask:255.255.255.0
          inet6 addr: fe80::6a05:caff:fe42:ffb9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:481605 errors:0 dropped:12416 overruns:0 frame:0
          TX packets:279469 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:103081957 (103.0 MB)  TX bytes:57792580 (57.7 MB)
          Interrupt:16 Memory:fb7c0000-fb7e0000
 
enp10s0.100 Link encap:Ethernet  HWaddr 68:05:ca:42:ff:b9
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::276a:311d:cad3:e2e3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1386 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2743 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:182972 (182.9 KB)  TX bytes:384873 (384.8 KB)
 
enp10s0.200 Link encap:Ethernet  HWaddr 68:05:ca:42:ff:b9
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::22e5:811a:a696:8555/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:89774 errors:0 dropped:12 overruns:0 frame:0
          TX packets:98005 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21732307 (21.7 MB)  TX bytes:23167350 (23.1 MB)
 
enp10s0.300 Link encap:Ethernet  HWaddr 68:05:ca:42:ff:b9
          inet addr:192.168.3.1  Bcast:192.168.3.255  Mask:255.255.255.0
          inet6 addr: fe80::f742:d553:67e6:ee47/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1308 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2552 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:166806 (166.8 KB)  TX bytes:364560 (364.5 KB)
 
ens6      Link encap:Ethernet  HWaddr 68:05:ca:42:ff:99
          inet addr:192.168.178.47  Bcast:192.168.178.255  Mask:255.255.255.0
          inet6 addr: 2a02:810d:8ac0:26c4:bd55:9449:7d8a:8b2d/64 Scope:Global
          inet6 addr: 2a02:810d:8ac0:26c4:1859:66a9:7f2a:aa5a/64 Scope:Global
          inet6 addr: fe80::302a:2822:b274:9819/64 Scope:Link
          inet6 addr: 2a02:810d:8ac0:26c4:391e:a044:e8eb:40a3/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1111548 errors:0 dropped:283 overruns:0 frame:0
          TX packets:522025 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:790240719 (790.2 MB)  TX bytes:101741848 (101.7 MB)
          Interrupt:27 Memory:fbbc0000-fbbe0000
 
ens6 is vmnet0 which is vmnic0 -> vmk3
enp10s0.100 is vmnet2 which is vmnic1 -> vmk1
enp10s0.200 is vmnet3 which is vmnic2 -> vmk0
enp10s0.300 is vmnet4 which is vmnic3 -> vmk2
 
root@bwell:~# vmware-networks --status
Bridge networking on vmnet0 is running
Bridge networking on vmnet2 is running
Bridge networking on vmnet3 is running
Bridge networking on vmnet4 is running
All the services configured on all the networks are running
 
root@bwell:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.178.1   0.0.0.0         UG    100    0        0 ens6
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 enp10s0
192.168.1.0     0.0.0.0         255.255.255.0   U     400    0        0 enp10s0.100
192.168.2.0     0.0.0.0         255.255.255.0   U     400    0        0 enp10s0.200
192.168.3.0     0.0.0.0         255.255.255.0   U     400    0        0 enp10s0.300
192.168.4.0     0.0.0.0         255.255.255.0   U     100    0        0 enp10s0
192.168.178.0   0.0.0.0         255.255.255.0   U     100    0        0 ens6
 

Jul 28 10:36:50 switchff3a2d 3SWCOS %3SWCOS-I-LOGACLINETPORTS: gi1/1/2: permit ACE IPv4(UDP) 0.0.0.0(68) -> 255.255.255.255(67),trapped
 
thats what comes up on the switch syslog when i try and get a dhcp address on either interface of the ubuntu clilent.
 
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPDISCOVER from 00:0c:29:0b:ca:be (usrvtest1) via enp10s0.200
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPOFFER on 192.168.2.8 to 00:0c:29:0b:ca:be (usrvtest1) via enp10s0.200
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPDISCOVER from 00:0c:29:0b:ca:be (usrvtest1) via enp10s0.200
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPOFFER on 192.168.2.8 to 00:0c:29:0b:ca:be (usrvtest1) via enp10s0.200
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPDISCOVER from 00:0c:29:0b:ca:be (usrvtest1) via 192.168.2.254
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPOFFER on 192.168.2.8 to 00:0c:29:0b:ca:be (usrvtest1) via 192.168.2.254
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPDISCOVER from 00:0c:29:0b:ca:be (usrvtest1) via 192.168.2.254
Jul 28 10:41:00 bwell dhcpd[12197]: DHCPOFFER on 192.168.2.8 to 00:0c:29:0b:ca:be (usrvtest1) via 192.168.2.254
 
discover offer but no answers from the dhcpd thats strange too. dhcpd works and answers normal when not coming from the esxi/workstation12 host.

vmware setup

if you need futher information to help me please answer. thanks
0
Comment
Question by:Member_2_7969923
  • 2
2 Comments
 

Accepted Solution

by:
Member_2_7969923 earned 0 total points
Comment Utility
i could solve it!!!!!
 
the only problem i still had was that the promiscuous mode on the vmnet devices could not be set because i was running the workstation as user not as root.
 
so what i did was ..
 
root@bwell:~# ls -al /dev/vmnet*
crw------- 1 root root 119, 0 Jul 27 04:21 /dev/vmnet0
crw------- 1 root root 119, 2 Jul 27 04:21 /dev/vmnet2
crw------- 1 root root 119, 3 Jul 27 04:21 /dev/vmnet3
crw------- 1 root root 119, 4 Jul 27 04:21 /dev/vmnet4
crw------- 1 root root 119, 5 Jul 29 07:21 /dev/vmnet5
root@bwell:~# chgrp baum /dev/vmnet*
root@bwell:~# chmod g+rw /dev/vmnet*
 
and now all the custom tcpip stacks can communicate with the vlan world.
 
so not only setting promiscuous mode on the esxi server is important.
0
 

Author Closing Comment

by:Member_2_7969923
Comment Utility
thanks to myself ;-)
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now