load balancer - website takes long to respond after failover

Im trying to set up two Load balancers, LM-1500 from Kemp technologies.
The setup was actually pretty easy to configure. The load balancer are configure in high availability.

The HA and failover for the Web Server configure in the virtual services work like a charm, but for the Load balancer failover, It doesnt seems to work properly.

For instance, if you power off the active load balancer, the other active/passive become the active one in less than 2 sec, so that part is good.

But while the failover works it still takes one minute for the website to respond. Sometime the website will start responding only when the actual active load balancer goes back up.

It seems like the switch (Cisco catalyst 2950 v12.1) does cannot clear the Mac address table fast enough. Ive tried to disable it or to put the Mac-address-table aging-time to the minimum (10) but still doesnt work.

Any idea would be appreciated


Who is Participating?
Aaron StreetConnect With a Mentor Infrastructure ManagerCommented:
sounds like you have the swich set (or soem thing set) to ignore gratrious ARP requests.

where a braodcast is sent out to the network to update the ARP cache of devices with a new IP address for a recoreded IP.

whqat should happen is when the fail over occours, the now primary device send out a broadcast to the network saying ip address xxx. is now held by this mac addresss (its own mac) thisshould over ride any MAR address CAche on the local devices and flush them with the new value.

Of course this is a security risk! as I could come alng, plug in my laptop and send out a garatruious arp request saying the ipaddress (the default gate way address for instance) it held be the mac address xxx.xxx.xxx.xxx (my laptop address) now all clients would point to my PC for the default gateway...

So often it is disabablesd for security reasosn. so make sure that devices are able to accept these types of requests. you should be able to limit only garatrious arps from limited stations. So only your load balanacers should be able to send these requests
How does the LM-1500 work with the virtual IP?  That is, does the backup one use the same MAC or a different MAC from the primary one?

If it uses a different MAC, then you need to see what the ARP table looks like in the Web server.

If it uses the same MAC, what do you see if you do a show mac-address-table is the MAC still in the table and it is still pointing to the old port?

Aaron StreetInfrastructure ManagerCommented:
Yes of course, you need to check the switchs are updating there mac tables correctly if it is a constant mac address used betweeen the two lqoad balanacers.

if you do a show mac addresstable on the switch. before and after a fail over you should see the address swap between the two ports

show arp  would show if the mac address assigned to the ip address is changing
Um, the more I think about it, the more I need to understand about how this load balancer works.

We use BigIP F5's and we to a NAT on the back side.  That is the inbound IP address from the end users is NAT'ed to a single IP address when the traffic goes to the backend server.  Each F5 uses its own unique IP address for this, so if F5A is active the backend server sees IP address #1, when F5A fails and we switch to F5B the backend server sees IP address #2.

Now if you are not NAT'ing like above, that means your backend server sees the real IP address and thus should be passing the traffic to a router.  Is your load balancer the "router" in this case?  If so, then we still need to know how the IP addressing on the "back side" (the one on the back end server side) is setup.  If you have a single IP address that flips between the two, then you do need to look at how your Cisco switch is handling the mac-address-table and arps.  If you have two unique IP addresses, then you should be doing some type of dynamic routing protocol to change the route table.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.