Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

Network or BigIP Load Balancer problem

Posted on 2006-10-19
8
554 Views
Last Modified: 2008-01-09
Hi,

We are having serious network problems and we're basically trying to pinpoint the problem.

Our LAN consist of :

2 failover PIX firewalls
2 unmanaged switches
15 Windows 2000 servers
1 BigIP load balancer
2 Load balanced web servers

The problem is that we're getting intermittent network connectivity between the servers.
I have analyzed traffic on a few servers and i'm getting a lot of TCP packets that has to be reassembled. (TCP segment of a reassembled PDU)

I'm also getting the following log entries in our load balancer that sits between the web servers and the other servers.
arp info overwritten for 10.1.88.38 by .....

We're at a point where we really don't know what to do and we have had 7 people working on this for a looooong time now. If anyone knows anything that could help me I would be reaaaly greatful!

Thanks!
0
Comment
Question by:entronet
  • 3
  • 3
8 Comments
 
LVL 57

Expert Comment

by:giltjr
ID: 17771248
---> arp info overwritten for 10.1.88.38 by .....

Some other box either believes that it is 10.1.88.38 or is configured to do proxy arp for that address.  Do you happen to have two boxes that have this same IP address.

--> I have analyzed traffic on a few servers and i'm getting a lot of TCP packets that has to be reassembled. (TCP segment of a reassembled PDU)

Sounds like you have a server that has a NIC that supports packet segmentation and has that option enabled.   Generally this is an option on Gigabit NIC's.  Do your servers have Gigabit NIC's?
0
 

Author Comment

by:entronet
ID: 17775410
I have found out that the ARP issue is a serious problem. What happens is that my servers ARP tables get overwritten with the wrong MAC address. The wrong MAC address is the MAC address of the PIX firewall. I have no idea how this is happening.

Some servers have Gigabit NIC's. So packet segmentation should be disabled on all the Gigabit cards?
0
 
LVL 57

Accepted Solution

by:
giltjr earned 500 total points
ID: 17775963
Does the PIX have this IP address coded on it, the 10.1.88.38?  Do you know what this IP address is supposed to be?

Well packet segmentation is a funny thing.  It reduces CPU on the sending box, but increases CPU on the receiving box.  If you have a sever that is talking to 100 clients.  You will reduce the CPU utilizaton on the sever, say by 5%, while increasing the CPU utilization on each client by say 0.0001 %.  these are made up percentages just to try and make a point.

Say you have 14,600 bytes to send out to each of the 100 clients.  

Disabling segmentation offload means the server's CPU will be used to break down the 14,600 bytes into 10 packets each in its own Ethernet frame (so 10 frames).  It must do this 100 times because there are 100 clients.  Now each client receives 10 Ethernet frames with one packet in each.

With segmentation offload enabled, the servers CPU just sends 14,600 bytes for each client.  The NIC will break these down into 10 Ethernet frames/IP segments of  1,460 bytes each, thus save CPU on the server.  Each client will still receive 10 ethernet frames that contain the segmented IP packet and will use a bit of more of their CPU to re-assemable the 10 Ethernet frames into 1 IP packet.

So, disabling segmentation could increase CPU utilization on the server.

You may want to read:

     http://en.wikipedia.org/wiki/TCP_segmentation_offloading

Also

     http://en.wikipedia.org/wiki/TCP_Offload_Engine

In case your NIC's have TOE.
0
Portable, direct connect server access

The ATEN CV211 connects a laptop directly to any server allowing you instant access to perform data maintenance and local operations, for quick troubleshooting, updating, service and repair.

 

Author Comment

by:entronet
ID: 19289740
Let me just add to this: If i didn't necessarily had to upgrade the app to be a Visual Studio 2005 program, could i more easily upgrade it to be compatible and use 2000, XP, Vista features?
0
 
LVL 57

Expert Comment

by:giltjr
ID: 19292099
What?  How does Visual Studio fit into this?  Maybe you updated the wrong question?
0
 

Author Comment

by:entronet
ID: 19292378
hehe, yes sorry :)
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Downtime reduced, data recovered by utilizing an Experts Exchange Business Account Challenge The United States Marine Corps employs more than 200,000 active-duty Marines with operations in four continents, all requiring complex networking system…
Greetings, Experts! First let me state that this website is top notch. I thoroughly enjoy the community that is shared here; those seeking help and those willing to sacrifice their time to help. It is fantastic. I am writing this article at th…
Two types of users will appreciate AOMEI Backupper Pro: 1 - Those with PCIe drives (and haven't found cloning software that works on them). 2 - Those who want a fast clone of their boot drive (no re-boots needed) and it can clone your drive wh…
A short tutorial showing how to set up an email signature in Outlook on the Web (previously known as OWA). For free email signatures designs, visit https://www.mail-signatures.com/articles/signature-templates/?sts=6651 If you want to manage em…

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question