802.3ad Static Link Aggregation Problems - One to One Scenario not working as planned

Finally, I would suggest using 10Gig ports as a solution...
At present time 10G ports are relatively cheap (especially CX4).

Romans

Hey Romans,

Good comments.  Well, I've learned more than I want to know about LAGs recently (unfortunately).  I read an engineering document (ironically I think one you posted before but I found the link elsewhere) from engineers at Broadcom and Intel - and they mentioned that LAGs were only effective when using multiple TRANSFERS.  They did not mention the limitation when going to only one IP or MAC on the destination end.

Any rate, that is good info about the multiple IP's that you put in there.

Here are two scenarios am proposing:

1) Luckily, our database/application software is Visual Foxpro based - so it only uses a regular Windows MAPPED drive to communicate from Terminal Server to Database server (mapped by IP).  So I DO have the option of writing a script and putting different store users logins into one group ... which say then runs a script and maps their drive like so:  M: drive mapped to 10.10.15.10 (one ip on Main Databse server).  Then say for a second group of people, I map their M: drive to a second Ip (on same NIC as you suggested) to say 10.10.15.11.

This would effectively use the aggregated link the way we want - although not 100%.  It's not TRUE load balancing of course since let's say we take the first half of stores (say 14 stores) .. and map them to the first .. and then the last 14 and map them to the second ... if the first 14 saturate their IP link ... and the second IP (link) is not saturated but not being used ... then it's somewhat of a moot case.

so ... onto your 2nd solution:

2)  Ironically, you and I think alike.  My Dell rep yesterday was thinking 4GB Fiber ... however he didn't realize 4GB Fiber is only for Backplanes it seems.  It looks like you can't connect these directly from server to server - and have to have a fiber switch plus fiber modules (2) for each connection - so the cost soared to over $3000 or so ($1500 4GB fiber cards - $800 or so on the fiber switch - plus guestimating $300 or so per fiber module).

So that puts us looking at 10GBe .. which ironically I looked at this morning after receiving his email about the 4GB problem.  Looking at the Intel Dual Port 10GBe PCI-Express Card (10 Gigabit AF DA Dual Port Server Adapter) which is copper based - and only running around $599 street price right now!  We need two of these so for $1200 vs $3000 on the 4GB fiber solution, a no brainer.  Plus we'll have an extra 10GBe port available for future expansion.

Question:  On your solution of making the 4GB link agg work - what kind of IP sharing (load balancing) solution would we have to use?  Some Microsoft-based version on the server (I'm guessing not since they don't even support Link Agg natively) ... or a hardware-based?  We actually just bought a Sonicwall 2400 which has load balancing - so I'm not sure if that would work or not.  We bought it for balancing the WAN connections though and not LAN - and at around $2800 (with support contracts and all), we'd need another to accomplish this goal.  Just makes the 10Gbe sound better and better by the minute.

Hi kmruss,

yep, possibly we think alike in some cases!
If you can interconnect two servers using copper cable (lenght for 10Gbps is vital) use 10G option!

However, you can also take 1st solution to save money and to use equipment you already have. In this case load balancing options on teamed ports should be set to L3 (by IPs) and configure 4 IP addresses for that teamed interface. Then map all four to terminal server and you will be able to use more than 1-2Gbps.
However, your switch should support L3 load sharing also, otherwise you will not archive good result.

Btw, there is one more option (if your switch is capable of L2 load sharing only):
you can throw away teaming idea at all. just use 4 different nics in both servers, pair them into different subnets and map drives to different IPs.

Unfortunately, I can't comment on Sonicwall option... not an exp there.

Hey Romans,

Good options.  However, in the Intel Proset, I don't see any options for telling it to load balance via L3 (by IPs) ... especially since it's using 802.3ad standards.  However, maybe that works with the DYNAMIC load balancing (with LACP).  And that's the trouble I've been having ... finding GOOD information that describes the advantages of Dynamic over Static LAG - and using LACP.

From what I've found so far, it describes LACP as basically only helping find configuration errors with the LAG groups - and some help in distributing.  I have found nothing solid to confirm that balancing by IPs is actually possible with 802.3ad - as I'm thinking it's not.  If you can find something, please let me know.  It actually sounds like what you are saying is to find a L3 switch capable of load balancing with IP's ... and forget the whole LAG idea.

However, problems still crop up with the scenario I proposed earlier - which is that those 14 clients have to be mapped to one IP .. while the other 14 are mapped to a second IP on those nics.  The first 14 could still saturate their link and the second 14 still have plenty of bandwidth left not being used.

By the time we find (if it's possible) and get a L3 capable IP load-sharing device, we could have paid for the $600 ($1200) Intel 10GBe Dual Port server nics 2 times over I'd bet.

I'm starting to think the Link Agg is just not worth it and too costly for our case.  It looks like it would work great with multiple destination Ip's ... but not good when going from one IP to another (one to one scenario with our servers)?

Hey Romans,

Sorry for the very delayed response - cleaning this up:

What we did was just stick with regular Gigabit networking - but went with a 4-port (QUAD) Gigabit Networking Card from Intel that supported Link Agg.  We're basically just sticking with 1-Gig interlink between servers right now because of the extra complications and lack of benefits versus hassle right now from doing link aggs.  I got it to work (link agg) - but the bottom-line was, our software company FINALLY admitted they had a problem in their software causing most of the 'bottleneck' in retrieving data from one server to another.

Your original recommendation of the 10Gig CX (Copper) interlinks was I believe the way to go - however here's what we found out when trying to implement that:  The MAIN problem - and this is going to sound hard to believe - last year in Aug 2008, EVERYWHERE was hard to find the SPECIFIC 10Gig CX Interlink cable you needed to interconnect the two cards together.  You CAN NOT use regular Cat5e or Cat6 cable - it is a special cable even though not fiber.  In addition, what REALLY put the price out of the ballpark is, I verified DIRECTLY through Intel that you COULD NOT do a 'cross-over' type connection with the 10Gig - or at least no one had come up with a 'cross-over' cable for it.  So that put us having to buy a 10Gig capable SWITCH - which is mainly used for backbones.  So intead of a cheap alternative in having say two 10Gig cards for around $400 to $600 per card ($1200 total) - it went to over $3000-$4000 considering you had to have the expensive $2000+ 10Gige Capable network switch to interconnect the two cards/servers.

So long story short, it wasn't worth the extreme cost to try to 'work around' the software companies network/bottleneck issue that they finally fixed for us.  We may consider 10Gige in the future and I'm glad I researched it and that you recommended it - but wasn't worth it for now.

Thanks Roman - awarding points.

Thanks Romans - you can read my response in the solution - points awarded!