What could drop OCI and transport connections on the network but not the link connection on the NIC?

What could drop OCI and transport connections on the network but not the link connection on the NIC?

      

Here is the deal.  This has me stumped.

Server a has a Broadcom NIC linking at 1Gps to a cisco switch.  We noticed over the week the OCI oracle database connection was dropping, we then notcied that HTTP connections and pc anywhere connections were also dropping at the same time on server a.  However, we see no packet loss or drops on server a NIC, no collisions, no packet  loss on the switch.  The NIC is not loosing connection to the Switch.  When we set the NIC to 100Mbps the drops go away go away and the link is stable.

So if i set server a NIC to 1Gbps, and hit the switch with heavy traffic, ping packets 8000bytes and then work on Server a it starts to drop connections on the apps, pc anywhere, Oracle, http but the NIC shows fine.  no packet loss on the nic or switch no collisions and no drops.


There was a firewall changed on the network however this firewall does no intercept traffic between clients connecting to server a.  What could cause the OCI oracle drops and the other application drops on server a at 1Gps that does not show up with 100Mbps link and also not show up with dropped packets, collisions or connection renegotiating?

this one has me stumped.   Obviously we want to keep the link at 1Gbps.
YankeeFan03Asked:
Who is Participating?
 
lrmooreConnect With a Mentor Commented:
This type behavior sometimes happens with a duplex mismatch between the nic and the switch.
Are they both at "auto" or is the server set for 1000/full-duplex and the switch auto?
They both need to be the same. Either both auto or both set to 1000/full
Do you see any error counters on the switchport?
It could also be that your big packet loading is interfering with the oracle application causing packet timeouts, but not dropped packets.
Is there anything else between this switch that the server is connected to and the users? Any other switches or routers or WAN links?
http://www.wireshark.org  <== packet sniffer. Look for retrans packets, "ack too long" and other window size issues.
http://www.solarwinds.net <= get the 30day eval of Orion, set up SNMP on the server and Orion on another workstation closer to the users. Monitor the server, to include CPU utilization and let it run 4-6 hours and see what your average packet delay times are. Also setup SNMP on the switch and monitor the port that this server is connected to.
0
 
lrmooreCommented:
Assuming that this application has been running for a good while with no issues, and this is a new problem, the obvious question is - what changed? MS updates on the server? IOS upgrade on the switch? New firewall - what else? What kind of firewall? How is it different from the old firewall?
0
 
YankeeFan03Author Commented:
yeah, this is on a customer network.  so all i know is what they tell me.  They instist there is only a switch between the client and the server.
the nic only has auto and a 100 full or half setting.  with it set to 100 full it works auto for the 1Gbps the the application layer seems to blow up.
the customer says they see nothing on the switch.
how could i check for packet timeouts?  what would i look for?  
the drops seem to occur at the 1G connection when we start large pings.  but it does not jsut drop the oracle connection but http and other apps as well.
what would i look for on the packet sniffer?

0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

 
pjtemplinConnect With a Mentor Commented:
Hmmm...duplex mismatch should be obvious.  The half-duplex side will show tons of late collisions, and the full-duplex side will show runts and FCS errors inbound.
0
 
lrmooreCommented:
>the customer says they see nothing on the switch.
Can you get switch make/model? Is is a managed switch or unmanaged? Is it Gigabit capable?
With the packet analyzer you can filter out results to just traffic to/from that one server and click Analyze and check for Warnings. Beware that you'll have to have a SPAN/Mirror port on the switch that the server is connected to.


0
 
YankeeFan03Author Commented:
I will try and get the switch specs.  i know it is a cisco blade switch that handles 10 100 1000 links.
I know it is managed.
I have been researching and it appears the TCP packets are breaking at the transport layer.  OCI, HTTP, i believe operate at this layer.  Are there some opbvious things on a packet sniff or switch that would verify this?
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.