Cisco 2811 configuration options.

lrmoore,

Yes we already use BGP so I will move that as well. So per packet load sharing should work well on both cards then and it gives me the option to add a 4th T1 on each side in the future if needed. I am hoping to load balance between both lines for a total of 9 mpbs. Does this seem feasible? I will have 4 WS-C3750-24TS-E layer 3 switches as well as a pair of PIX 515e's unrestricted. I have 2 DMZ's as well as a my Cisco net for the WAN. I have one /23 now and should be able to get another /23 from Verizon. I am just in the designing stage now so that is why I am bouncing these things off of everyone. Is it possible to reliably load balance this configuration? I am more concerned about outbound traffic since I host some pretty heavy traffic websites.  

Yes, to the CEF per-packet on up to 4 T1's.
BGP will help you load-share between the two (notice I did not say load-balance) "sets" of T1's.
However, with two different routers, this is more difficult.

For load-balancing you have a couple of issues
1) the PIX firewall can only have one default gateway. Good news is that you can use OSPF between both routers and the PIX, and redistribute a default. The PIX can also have a "backup" route.
2) Are you planning the PIX's in a active/standby failover pair configuration? Do you have them yet? If not, consider the newer ASA5520's
3) With two different /23 network blocks, which block will be your primary? You may have some issues with inbound NAT with 2 different public IP's to 1 server..

There are other options, just depends on what you already have vs what your intent is and how much experience you have setting up this type network operation.

I do have both PIX's already with the latest image planned in a active/standby configuration. What I am hoping to do is to multi home the web servers on separate net blocks and load balance traffic. I am most concerned with outbound from the web servers. I have a consultant that is helping with that portion but I like to be familiar ahead of time so when he leaves I am not lost if something were to fail. I was thinking I could carve a /24 out of each of the /23's for the DMZ and a /25 or so for the Cisco out of any of the others. We plan on OSPF as well for for the switches as well but am not familiar enough yet to understand completely that scenario. How does that sound? possible?

It does sound possible, but I'm not so sure about dual-homing the servers. The issue is that the server can only have one default gateway and will not load-balance if multiple NIC's are in different IP subnets.
The best scenario would be for you to have your own IP address space and BGP AS# and advertise that IP subnet out both ISP's via BGP instead of getting 2 different IP blocks and having to figure out how you can nat 1 IP from each block back to the same server

I do currently have my own AS# and a /23 ip block. The only NAT'ing I had planned on doing was for the LAN. I had just planned on creating a small pool out of one of the subnets. So you are suggesting sort of a round robin for the servers within the same subnet? Can you recommend a good book that has some WAN topology examples? So you see what I am getting at. I am hosting center and am mainly concerned about efficiently giving my customers the most bandwidth available while at the same time providing redundancy.

BTW........ thanks for all your help with this.

If your primary concern is efficiently giving your customers bandwidth, DON'T NAT.  Especially if you have a firewall, don't fuss with NAT.

Ok so I can do multilink. but I am not sure what you mean about RRAS? Is this similar to the loopback principal in BGP?  What I am most concerned with is taking advantage of  both 4.5 mbps lines to give my web servers 9 mbps while still having redundancy.  

No, NAT was never an option for the web servers, only for my developers for internet access.

RRAS is Windows' Routing and Remote Access Service.  It lets you run OSPF on Windows servers.

You said you had two PIXen in active/standby.  Figure out your network topology, and then decide if you need two default routes at the server level to achieve your goals.  My guess is you don't - you want two default routes on the PIXen.

If your biggest concern is balancing the bandwidth, you might want to look at something like content server switch/hardware load balancer.
There is even a module that you can put in your 2800 routers http://www.cisco.com/en/US/products/hw/modules/ps2797/products_data_sheet09186a008010fb9f.html

DNS round-robin is only a partial answer. Distributed director is more like DNS on sterioids that can resolve based on several criteria, not just RR.
Distributed Director is an IOS feature . .
http://www.cisco.com/application/pdf/en/us/guest/products/ps813/c1244/ccmigration_09186a008007ca69.pdf

I am trying to draw this out and make sure I understand these options..... this is a lot to take in. Thanks guys. I will get back to you.

I understand what RRAS is but was curious about what the loop back is for in the.

 Comment from pjtemplin
Date: 12/14/2006 07:47AM PST
      Comment       Accept

RRAS is Windows' Routing and Remote Access Service.  It lets you run OSPF on Windows servers.

Sample:

Router 1:
LAN: 1.1.1.14/28

Router 2:
LAN: 1.1.1.30/28
(Assume Router 1 & Router 2 interconnect somehow; the method isn't relevant at the moment.)

Server1:
NIC1: 1.1.1.1/28
NIC2: 1.1.1.17/28
Lo1: 1.1.1.33/32

Server 2:
NIC1: 1.1.1.2/28
NIC2: 1.1.1.18/28
Lo1: 1.1.1.34/32

OSPF active on all interfaces.

server1.mydomain.com is 1.1.1.33.
server2.mydomain.com is 1.1.1.34.

Steady state, server1 advertises 1.1.1.33/32 over NIC1 and NIC2.  server2 advertises 1.1.1.34/32 over NIC1 and NIC2.  Either router can send traffic to 1.1.1.33 and 1.1.1.34 directly.

Failure: server1 NIC1 goes down.

OSPF loses its adjacency between router1 and server1 NIC1.  OSPF still has reachability through router2 and server1's NIC2.  1.1.1.33 remains reachable, but only through router2 instead of through both routers.

Had you used the NIC addresses in DNS, particularly NIC1's address, it would be DOWN.  Because you used the loopback address, it remains reachable after OSPF reroutes.

Ahhhhhh Thanks........ that just hit me in the head like a hammer. Now I understand.

So here is my last question to you guys... I think I am almost there.

1) How do I support virtual IP's on the webservers with the example used above. Ex... when I have a site with a cert it has to have it's own IP because the header requests are encrypted so host headers won't work.

2) lrmoore, how do I influence my outbound traffic to gain the full 9 megs. I am only concerned about outbound traffic because I host web servers. Lastly, can I incorporate my LAN into this environment. My developers would need access to the internet as well.

Assign more addresses to the MS loopback adapter or whatever loopback you're using.

Tell us how/where the farthest point inside your network where you're using two default gateways.  That's where you'll want/need to begin focusing on load balancing.

If outbound traffic needs to be more than 4.5Mbps, I'd strongly recommend getting 3xT1 from a third ISP.

Farthest point would be the DMZ's. So load balancing is handled by the servers. So if I am alternating between routers (If I understand this correctly) I will manipulate my routes with BGP route maps to ensure inbound traffic comes back in the same router? Correct?

As for T1's I am getting 3 per side for a total of 9Mbps.