Networking question with two LANs

oaklandgroup
oaklandgroup used Ask the Experts™
on
Hi There,
I have this really strange networking problem which is completely baffling me.

If need be I will draw diagrams or something but the problem I have is this:

I have two physical servers running Server 2008 R2. Both are member servers and have next to nothing installed (although one has the iscsi software target installed but not configured).

Both server's have two NIC's

NIC1 of both servers are connected to one switch
NIC2 of both servers are connected to another.

The switches are not linked

First switch is connected to the rest of the domain, the second switch only has a storage device on but this is not currently relevant.

Server1 NIC1 IP 192.168.1.2 with gateway .254
Server1 NIC2 IP 192.168.2.2 with no gateway

Server2 NIC1 IP 192.168.1.4 with gateway .254
Server2 NIC2 IP 192.168.2.4 with no gateway

Now the network is working perfectly through the first NIC's as both are on the domain happily and can pick up any resources else where on the network

Server1 can ping 192.168.2.4 (server2 NIC2) with no problems
Server2 can NOT ping 192.168.2.2 (server1 NIC2)
However Server2 can ping the storage device on 192.168.2.10

As far as I can tell, the static routes are all correct but for reference...
Server1:
===========================================================================
Interface List
 13...a4 ba db 40 13 8a ......Broadcom BCM5716C NetXtreme II GigE (NDIS VBD Client) #2
 11...a4 ba db 40 13 89 ......Broadcom BCM5716C NetXtreme II GigE (NDIS VBD Client)
  1...........................Software Loopback Interface 1
 12...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
 14...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0    192.168.1.254      192.168.1.2    266
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
      192.168.1.0    255.255.255.0         On-link       192.168.1.2    266
      192.168.1.2  255.255.255.255         On-link       192.168.1.2    266
    192.168.1.255  255.255.255.255         On-link       192.168.1.2    266
      192.168.2.0    255.255.255.0         On-link       192.168.2.2    266
      192.168.2.2  255.255.255.255         On-link       192.168.2.2    266
    192.168.2.255  255.255.255.255         On-link       192.168.2.2    266
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link       192.168.1.2    266
        224.0.0.0        240.0.0.0         On-link       192.168.2.2    266
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link       192.168.1.2    266
  255.255.255.255  255.255.255.255         On-link       192.168.2.2    266
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0    192.168.1.254  Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  1    306 ::1/128                  On-link
 11    266 fe80::/64                On-link
 13    266 fe80::/64                On-link
 11    266 fe80::54c4:5fa5:98c6:cb1/128
                                    On-link
 13    266 fe80::c1b2:c6da:f3b1:da7f/128
                                    On-link
  1    306 ff00::/8                 On-link
 11    266 ff00::/8                 On-link
 13    266 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None


Server2:

===========================================================================
Interface List
 16...02 11 43 dc cc 8d ......Microsoft Failover Cluster Virtual Adapter
 12...00 11 43 dc cc 8e ......Intel(R) PRO/1000 MT Network Connection #2
 11...00 11 43 dc cc 8d ......Intel(R) PRO/1000 MT Network Connection
  1...........................Software Loopback Interface 1
 13...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
 14...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
 15...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0    192.168.1.254      192.168.1.4    266
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
      192.168.1.0    255.255.255.0         On-link       192.168.1.4    266
      192.168.1.4  255.255.255.255         On-link       192.168.1.4    266
    192.168.1.255  255.255.255.255         On-link       192.168.1.4    266
      192.168.2.0    255.255.255.0         On-link       192.168.2.4    266
      192.168.2.4  255.255.255.255         On-link       192.168.2.4    266
    192.168.2.255  255.255.255.255         On-link       192.168.2.4    266
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link       192.168.1.4    266
        224.0.0.0        240.0.0.0         On-link       192.168.2.4    266
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link       192.168.1.4    266
  255.255.255.255  255.255.255.255         On-link       192.168.2.4    266
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0    192.168.1.254  Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  1    306 ::1/128                  On-link
 11    266 fe80::/64                On-link
 12    266 fe80::/64                On-link
 12    266 fe80::2906:39d2:db74:fd91/128
                                    On-link
 11    266 fe80::307c:8f24:6612:db02/128
                                    On-link
  1    306 ff00::/8                 On-link
 11    266 ff00::/8                 On-link
 12    266 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None


I have ruled out the firewalls on both servers and the switch is not a managed switch of any kind with any restrictions.

I'm tearing my hair out over this one and I am convinced all I need is a second pair of eyes to point out something obvious.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Author

Commented:
I have also ruled out any hardware or cabling/switch issues
Senior IT Advisor
Top Expert 2008
Commented:
OLn the switch where both 192.168.2.x nics are connected check the ports.  It's strictly a layer 2 issue since both are on the same broadcast domain.  Check the ports ont the switch for speed/duplex.

TO get the switch out of the picture take the  2 ports ont he 192.168.2 network and connect via a crossover cable, this will isolate whether switch or nic settings.

Author

Commented:
using cross over cable between servers has made no difference
Paul SolovyovskySenior IT Advisor
Top Expert 2008
Commented:
then you know that it's not the switch. Routes shouldn't matter in this case since it's on the same layer 2 broadcast domain, no routing is required.  At this point it's an OS/Nic driver issue.  

Commented:
@Paulsolov:
Correct me if I am wrong, I think he stated the two nics are Physically separated. So, to ping the 2nd nic on the server that is physically separted RRAS must be installed and you are routing over the server.. In the other case, physical separation prevents you from communicating on the 2nd NIC. Just a thought:

Author:

It is also NOT advisable to multihome a computer under any circumstances. The reason being is it registers both nics in DNS, (hence messing up DNS), It also causes Netbios to bind to one nic, and sometimes you have to change the nic binding. Also the gateway is a problem, and DHCP is a problem.. Since DNS, DHCP, Netbios, and the ARP table mess up with multihoming a server, it usually messes up your entire domain by having replication problems in FRS, DNS metadata, a bad ARP cache, Netbios Bind issues and therefore problems with the domain master browser services. The only times I would multihome a server is for Load balancing (with properly configured load balancing software), or to route over the server (which is not needed if you have a router: Router is preferred because of the amounts of processing it takes).
Paul SolovyovskySenior IT Advisor
Top Expert 2008
Commented:
@ChiefIT: Not sure..seems that the second switch is only used for 192.168.2.x network and connecting both nic via xo cable didn't work. Also loooks like the prod nics are also on the same subnet.  I could be reading it wrong..wouldn't be the first time.

First switch is connected to the rest of the domain, the second switch only has a storage device on but this is not currently relevant.

Server1 NIC1 IP 192.168.1.2 with gateway .254
Server1 NIC2 IP 192.168.2.2 with no gateway

Server2 NIC1 IP 192.168.1.4 with gateway .254
Server2 NIC2 IP 192.168.2.4 with no gateway
Commented:
@paulsolov:

Yah, I think we need a bit of clarification. I still recommend not multihoming a server. Nothing good comes of it.
Paul SolovyovskySenior IT Advisor
Top Expert 2008
Commented:
@ChiefIT

My impression was the the second nic was only to be use for storage (iscsi I assume) which would make sense.  Otherwise multihoming is nothing but problems and traffic should be routed in that case.
Commented:
@Paulsolov:

I see the intent of the author to have a separate LAN for NAS storage. However, the multiple nics would cause problems with domain controllers, (unless configured exactly right).

Even on two physically separated networks,

--meaning on nic for the NAS, and one on the LAN

then, two active nics still causes problems with the netbios bindings, DNS, and ARP cache.

With two active NICs (on the same subnet and physically separated especially), I could see much more severe problems than ping.. The author must be pinging in order to troubleshoot another problem. Otherwise, why ping?

I really think the second NIC IS the problem the author is seeing but not relenting to.

Author

Commented:
The intention is for iscsi eventually and I think where this could be falling over. I am currently waiting on a third NIC to be installed to test a couple of things.

Sorry for the delayed response
Commented:
So, I have to forwarn you of the issues caused when multihoming a domain controller.

Multihoming simply means having more than one IP address on the domain controller. It is usually used in one of two cases. One is routing over the server. The other is load balancing.

Routing over the server requires Routing and Remote Access. The two nics have to be on separate subnets. Then, ONLY one should have the default gateway, AND be the designated nic to register istelf within DNS. Otherwise, (without RRAS), you mess up the computer's routing table. Also, the computer will be confused on the default route (with two nics both having a default gateway). Also, the computer will register both nics within DNS and your server will not know what nic to provide DNS name resolution to it's own clients. Furthermore, Netbios needs to be bound to the internal nic when routing over the server. (NOTE: routing over the server IS NOT a good idea. routers carry large buffers and processors to store, manipulate, and route packets. This really taxes the server. So, use a router for what a router is for)

Load balancing requires very specific software configurations on the switch and the server. Load balancing is usually used to overcome a network cards choking point of number of simultaneous connections and requests. However, on a gig nic, properly configured, you should have no problems supporting 250 computers on a single server... If you extend beyond that, then consider a server cluster rather than load balancing. (There are a lot of problems with load balancing software. So, why put yourself in this position).

Your current situation requires that you have NAS storage on a separate network connection. Now, think about this>
You have two nics. One on the LAN, One on the NAS. Here are the problems you will see:

Default gateway:
If you configure both nics with a default gateway, where will packets default to, (if not within the routing table). So, you may experience intermittent network connectivity and the inability to ping.

DNS:
If you configure both nics to register its nic settings in DNS, then they both do. If a domain controller, then you have two sets of Host A records and Two sets of SRV records in DNS. Also, your computer may deem one nic busy when providing DNS resolution and spit out the answer to the DNS query on the wrong nic. Now, you have two problems. Clients and servers don't know what IP to go to for Domain SeRVices, AND your server doesn't know what nic to provide answers to DNS queries on.

Netbios:
A lot of folks think Netbios is dead. It's not. Netbios is used for RPC locator services and licensing services and the netlogon service. If Netbios BINDS to the NAS nic, then you loose netbios services on that computer.

DHCP:
Yes, I have seen people configure their server that is multihomed as a DHCP server. In fact, here is the post. DHCP providing IPs on the wrong adapters>

http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/2003_Server/Q_23806816.html

So, imagine this:
multiple nics cause problems with DNS, DHCP, routing tables, Netbios and therefor CIFS/SAMBA shares, etc....

I highly recommend not multihoming a computer without thoroughly reviewing this thread, and its warnings:
http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/2003_Server/Q_23806816.html
Commented:
first check if the network card 2 is failed on server 1, second check if failed port switch, those related to 192.168.2.22.

Next step, verify  the routing of multihomming on server. It is not advisable to use the server routing, is preferable to place another router to route these NICs 2 in each server.

Author

Commented:
Abandoned

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial