Losing Production Network Connection with Dual Nics on different subnets

Posted on 2009-12-28
Last Modified: 2013-11-09
I have a Systems Center Operations Manger R2 RMS server running Windows Server 2003 x64.  The server is an HP BL460c G1 blade with latest firmware/drivers.  

My network configuration is as follows:
2 Internal Nics
NIC 1: Prod network with 172.17.*.* IP
NIC 2: DISABLED in windows

2 Nics from Mezzanine card
NIC 3: GPRS Modem attached via Network with IP
NIC 4: DISABLED in windows

So in total, I have 4 NICs, 2 of which are in use.  NIC 3 is attached to a cellular modem to perform paging from SCOM.  

The configuration works.  My production network works fine and the paging works fine.  The issue that I have is that after a few days, my production network just stops functioning.  The NIC itself is connected, but I cannot ping the server what-so-ever.  If I connect via the iLO (remote interface) on the server, the server will be up and running.  As soon as I disable NIC 3 with the modem, my production network instantly comes back!

It seems like the production traffic gets confused and no longer tries to go through NIC1.  I can immediatley re-enable NIC3 (modem) and everything works fine again for a few days.  NOTHING is logged in the application or system log stating what happened.  There will eventually be events stating that the server can't communicate with a DC and you can tell when SCOM stops working, but no events state why the traffic stops communicating via NIC1.  

My Binding order is set in the following order:
1. Production Network
2. GPRSModem
3. Disabled NIC
4. Disabled Nic

Note: If I leave the modem NIC disabled, it will function fine and I have not experienced the issue.  

Other note: I completely removed the HP teaming software hoping that it might be it, but that didn't help either.  

Does anyone have a clue why this might be happening?  Please let me know if you need any additional information.  
Question by:STEHelp

    Author Comment

    I think I might have answered my own question, but I'd like someone to confirm.

    I just double checked my NIC configuration.  Here is the full config for each active NIC.  I had two different gateways specified!

    Production NIC:
    IP: 172.17.89.*

    Modem NIC:
    Default Gateway:

    I just cleared the default gateway on the modem NIC and I can still communicate with it just fine.  Could that have been the issue?  
    LVL 31

    Expert Comment


    It shouldn't be an issue, but it can be an issue.  While I am sure you would do this anyway, your best bet a this time is to monitor the machine now that you have made this change and see what happens.

    LVL 9

    Accepted Solution

    Hello STEHelp,

    Yes it's the default gateway causing the issue. When you are visiting an IP that your computer doesn't know where, it will be sent to the Default Gateway, which in your case is your localmachine. That's why you have the network issue. When you have 2 NICs, a best practice is to define the IPTables to make sure your computer know when to use the correct NICs to visit different networks. Your guess is correct. : )


    Author Closing Comment

    That makes sense.  I will close this for now and monitor.  

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    Scenerio: You have a server running Server 2003 and have applied a retail pack of Terminal Server Licenses.  You want to change servers or your server has crashed and you need to reapply the Terminal Server Licenses. When you enter the 16-digit lic…
    Network ports are the threads that hold network communication together. They are an essential part of networking that can be easily ignore or misunderstood, my goals is to show those who don't have a strong network foundation how network ports opera…
    Hi everyone! This is Experts Exchange customer support.  This quick video will show you how to change your primary email address.  If you have any questions, then please Write a Comment below!
    In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

    754 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now