How to aggregate the bandwidth of two cable modems?

Are you really using all that bandwidth? Also if you are using that much bandwidth comcast can shut down your account without warning if you are DLing lots of stuff or uploading. Also what is your network setup like right now and what devices do you have connected to it?

A normal switch will not be able to aggregate the links in that way, as your client devices can only have one gateway. What you need is for your gateway to allow aggregation or load balancing. Netgear make a decent little load balancing router, Snapgears are also good devices to do this. With the right device it is a trivial matter to connect two wan links, one lan link and away you go.

Novensiles thats what i was wondering.. Is if he had the devices to do this. The items you listed are managed or unmanaged? Assuming managed.

OK, so no switch, but the load balancing router.  Yes, I need the bandwidth.  I have about 1Tb to download, and the data is quite valuable; the sooner I get it, the more it is worth. Comcast tells me that so long as I have two accounts, I will get 2X the bandwidth from them.

Downstream of the rounter is a networked HDD (gigabit speed) and a few PCs.  Also a networked printer.

Also, I have another PC to connect to the router via wifi.  So in an ideal world a wifi loadbalancing router.

Looks like the dual-WAN stuff doesn't include wifi.  I think I could give my existing wifi router a different IP address and connect it as one of the clients of the dual-Wan unit, then allow at least the wifi PCs to connect through it.

Frankly, I've set up a lot of LAN stuff in my house, but it's been pretty self-configuring, plain vanilla stuff.  With this dual-Wan unit am I in for a lot of new challenges, or pretty much hook 'em up and away they go?

I hate to be the one to rain on the parade, but you're not going to see an increase in speed by using a load balancing router.

All those devices do is split the traffic flows over the two links. For example, if you had 10 users, the traffic of half would be directed over one link and the other half would be directed over the other link.

Or if you had one user running a web browser, email, telnet and IM. The traffic of two of the apps would be directed over one link and the other two apps would be directed over the other.

I your case, it sounds like all the traffic is being received by a single application.

Mike: A load balancing router requires a little more configuration, but with the devices I mentioned this is easily configured by a self-explanatory gui. If you understand the basics of IP you should have no issues, and if you don't the help files (at least on the snapgears) explain it in simple steps.
You can connect your wireless router into the LAN and have wifi devices connect through it, again fairly straightforward.

However I have to agree with donjohnston, you won't see double the speed for a single application unless it uses multiple connections ie. bittorrent.

OK, change in configuration.  Due to physical contrataints, we're changing the setup a bit.  We are downloading data from a remote site using multiple proxy servers.  The proxys all report to my in-house PC.  No data goes back out, only in.  So the proxys will connect to the PC, and that's where it ends; nothing more past the PC.  So no worries about wifi, etc, just aggregating the data stream into the PC.  Will this give me 2X the download speed?  Still need the load balancing router, switch won't work ,etc?  

If this will not do the deed, what about having two NIC cards in the PC, each connected to a different modem (each with a router to provide firewall function)?  

BTW: I may have mischaracterized my needs.  What I am after is not 2X the speed, but 2X the tonnage of data I can get down the wire.  As stated earlier, I have 1Tb to go get, and once I'm done I don't need to do this again.  So it's a matter of how long on the calendar it takes to get it.  So, not after 32Mbps out of a wire, but 16Mbps on two channels, the data all flowing into a networked HDD.

OK, my "net savvy" son tells me to point out that this is not a single terabyte file; it is several million files of about 200K each.

Still sounds like you need a load balancing router. You *could* use two nics and dual home your desktop, but if you use windows as a desktop OS you can only have one gateway and must add static routes for certain IP's in order to manually 'load balance' the connection.

The best way to achieve the result your after is to get the right hardware and load balance the links. This will give you twice the overall data transfer if the software you are using can make use of multiple simultaneous links to the data source. If the proxies are calling home you'll need to split up the two IP addresses they report to manually.(two wan connections = two external IP's)

A critical unanswered factor is the software you are using to do this download - can you provide any details?

PHP script on Linux OS.

So you have a (or multiple) linux webserver running a php script which sends data to multiple proxies which forward on to your pc which then stores the many little files in a directory?

Sorry, I'm not being clear.  There is only one PC involved; a Linux-based PC running Linux, which I am using as a server.  It sends commands, one at a time, to one of multiple proxy servers.  The proxy servers obtain the desired data and return it to the server, which gave the proxy the command to do so.  What I don't understand is how to somehow segregate proxys by-modem or to take turns / balance such that both modems are at work at the same time.  I'm starting to feel that the only way to get there from here would be to have two servers, a modem for each, and let them write the returned files to a common networked HDD.  Would like to avoid that expense.

Not sure how long Comcast would take, and assuming they'll not penalize you for pulling 1 TB down all at once...

At 10Mbps - it would take about 10 days to download a 1TB file. (I think I did the math correctly).

You'd likely have to place an order for the second modem, then wait for an installer to come out... not sure how time sensative this data is...  But if comcast is onboard for a once and done...

I'd honestly start downloading now with your single modem.

Loadbalancing is designed to 'split' traffic when large networks are involved.

Even if you have multiple downloads concurrently -- you can only fit so much data through your pipe (12Mbps). If you have a 6Mbps & 6Mpbs going on the same computer, 1 modem -- same time to DL as 1 12Mbps transfer.

For a once and done, I'd download the data, and not worry about second modems and the like.

Comcast stated that each modem would provide an independent 12Mbps to 16Mbps.  And note that it is not one big file, but a few million of 200K bytes each.  So not trying to dlownlad one big file, but a ton of very small ones.  So I just need one modem to be busy for a few seconds with one file and the othe busy for a few seconds with a different one.

What I keep coming back to is two NIC cards, no router, just each NIC connected to a different cable modem.  What I lack is the ability to select one NIC or the other to send out and receive from.  I believe gamers do something like this, in that I hve seen game PCs with two NIC crds built in.

Comcast is correct, each modem will deliver what they promise it too...
However, what everyone is saying is...
... for your scenario to work - its not that easy.

If you had 2 computers, 1 for each modem (even borrowing a friends laptop, if thats possible) would allow you to download from each modem independently.

It sounds like you need a managed switch capable of handling:

Link aggregation
Dual WAN

... You'd Team the WAN // Team the LAN

OS capable of Teaming NICs & the management software from the developer.

I'm not sure that the OS will support what you're trying to do natively, especially without the switch.

Yes, - PCs do come with 2, 3, 4 NICs built-in. they are used for different purposes... Proxy control (captive portal), load balancing, etc. I can't imagine there are many residential accounts that conduct load balancing across multiple broadband connections.

You can not aggregate the bandwidth of two cable modems with Comcast.  In order to aggregate the bandwidth between the two links it has to be done from the ISP end and Comcast does not provide this service.

Dual links work with certain Gaming and ISP services because they have a mechanism in place at both ends to support and aggregate the traffic between the two links; software at the customer end and at the ISP control and manage how packets are sent and received over each link. If you have control over the other end of your file transfer you could use a device like FatPipe at both ends to aggregate your link, but you need to use the device at both ends.

A load balancing router will not help your situation, neither will two NICs.  The way LB works in your situation is that when an application makes a connection the Router or OS (Dual NIC) will choose a link to establish the connection from.  If a different application makes a request, it will go out the second link.  Lets say you kick off an FTP session for file copy, it will be load balance to link 1.  If you then kick off a Telnet session, that session will be load balanced to link 2.  So your file transfers will only happen on one link.  

There you  go old_mike. I haven't used anything to multiplex different modems.

I think you're best bets are:

1) just download with your computer (thats what I'd do) ... 1 NIC
2) borrow a computer and buy a 2nd account, download with BOTH computers.

For a once and done transfer, what you are asking is really going to extremes.

With CLoz's information - I'll bet Comcast (if they are willing) will ask a hefty price tag for the multiplex.

Hello, Im the son of the author, and the one doing the setup. I thought it would be helpful if I pop in and clarify a few details here.

Currently we are setting it up as such:
One fedora server behind two separate 16mbps connections. What we want to do is agregate on the connection / request level So, while the fatpipe solution would be much smoother and aggregate at a packet level, as you say it would require a unit at both ends. While an ftp download of many files would be "glued" to one pipe, that is because there is only one connection for all the requests. What we have instead is many instances of php based scripts making separate requests without a persistent connection. Each of these requests establishes a new connection, and as such should be (in my mind) open to aggregation on the request / connection level.

As far as the actual unit doing the agregation, thats the pinch. We have looked at load balancing, but it seems that those routers balance the load between multiple computers, and are only on fail switches for switching internet pipes. My concern with two nic cards is that we are bringing the split back to the computer level, but still have not established a system to manage aggregation.

I have looked at a particular motherboard by XFX (nforce 780i sli) which has two onboard ethernet plugs, and boasts that it can manage it to produce twice the maximum throughput between a LAN. My assumption is that this is the same as dropping in two nic cards, with the exception that, as it was built specifically for this purpose, they have a built in driver which handles the aggregation. I dont know, however, if this
A. would work just as well if we were to plug directly both modems into its dual ports (or ran each modem to a separate router for the fw and then ran each of those to a port)
 or
B. if there is an alternative to accomplish the same thing.

Are these connections outbound or inbound request to the server?  It makes a big difference in what you can do.

To start, If you are not in control of both ends of the connection you cannot aggregate links, you will be load balancing.  These are two distinct functions.

If you are load balancing outbound request from the server then you can use a single Fatpipe or Linkproof from your location.  But be aware you can only load balance connectionless applications/protocols like HTTP.  Although you are using a PHP script to initiate a connection and file transfer, it's still factored on what application/protocol is actually executing the file transfers and whether or not the source and destination are the same for each request.  Which falls back to the issue of an application binding to a link.  

OK, thanks. The traffic is actually http requests sent BY our server to other servers, which is why it seemed to me that this would work, ie send a request, response comes back on the same pipe. There are no incoming requests, only outbound. Roughly what would a fatpipe or linkproof setup cost? I looked in on the websites and they looked like a solution for a rather large scale operation, not a small business scale, but they didnt mention pricing. I know channel bonding wouldnt work as it breaks the request at a packet level, is there no way to setup the OS to just flip back and forth sending one request down one pipe, the next down the other, and take the responses as they come in? (ie on two nic cards)

Edit: I thought I should clarify, all traffic is in the format of a php generated HTTP request (from one of many concurrently running php scripts) originating from our server and going out to one of many other servers, the response to which then comes back and is processed by the originating script. (typically the response is a roughly 200kB file which is then just saved).The load we need to aggregate is, of course, the response we recieve as it is MUCH larger than the request, and my assumption is that by aggregating one request down pipe A and the next down pipe B our responses would then come back on the pipes requested from, thereby splitting the load between the two connections.

Hope that helps.

Appliances that let you do this are usually start in the $3K to $8K range.  You might be able to find a refurb on an auction site like EBay.  

I'm not sure if you can do it at the OS level but one of the experts might have a solution.

If you would be willing, please take a look at:

https://www.experts-exchange.com/questions/21129358/dynamic-gateway-switching.html?sfQueryTermInfo=1+gatewai+switch

This level of stuff is a little out of my pond (hence asking) however it seems to me that this could be a workable solution, although it seems like doing surgery with a mallet. Am I right? the gist is to have a script which would switch the default gateway, and one to switch it back. Before each request is made, one of the scripts is run, effectively switching back and forth between gateways. Im curious, if I were to do this, the response should come back to the same IP the request went out on (and hence back to the same gateway), however if the gateways were swapped in between the request and response times, would the originating script still get the response?

Great! thanks so much for all the great help folks. I also just sorted out a way to do it at the application level. 2 nic cards, and, using php still, I first open a socket, bind it to the IP/port of the line I want to go down, and then go on as I was before. By specifying each binding I can send a request down this one, then down that one, etc. Seems to work. (if anyone else needs this, check out fsock_open, stream_context_create and stream_socket_client at php.net for details).

Noven, will snap gear switch on fail or each time a request is sent? the load balancing routers we looked at were only on fail and I would love to know if there is a real aggregating one along the small business (read $500 or less) level.

The snapgear 560 definitely does load balancing as well as failover only configurations (see the securecomputing.com website for full specs). We have it in use at a few sites and load balancing works quite well. It just scrapes into the $500 range (over here in aus anyway, probably cheaper if you're overseas).

Of course the geek in me says the php is a cooler solution, the initial post didn't imply application coding was an option ;)

Does the Snapgear offer packet base load balancing without source address persistence?  Many low end load balancers to simplify their code to offer a cheap LB by having a defacto source/destination address persistence load balancing.  What this means for example is that if Workstation A wants to make an outbound connection on port 80 (HTTP) the router will load balance that request and all subsequent request from that same workstation and port through the same link.  

Does the Snapgear allow the user to disable persistence?

What I want to make sure we're all considering is a somewhat different environment / need than others:

1.  Only one computer.
2.  Two cable modems, two separate accounts.
3.  Speed is not the issue; number of individual, small files is.
4.  We own the hardware (can add cards) and the software (can program with PHP, PERL, others if necessary).

So, may be the only way to go, but aggregating the speed at a packet level would work but probably takes more hardware.  The ideal would be to simply alternate or mix which modem through which requests go out, and that they return the same path and we be able to pick up the returns

Wow; 30+ entries to solve this!  Thummper (my son; lives nearby and we work together) was able to absorb all the suggestions and links and make a system that works well.  We now have two NICs, two routers, and two cable modems all managed by one Linux PC that is humming along well.  The key difference here compared to prior solutions was not to worry about increasing bandwidth per se, but "file width".  That is, we are downloading about 200K small files per day, alternating which modem through which they are fetched and brought back.  So same effect, but would not be useful for one wanting to download very big files faster.  Thanks to all.  Thummper opened up another question that lead to the final solution, and will post his results and code.