Solved

TMG ISP Redundancy with multiple IPs

Posted on 2010-09-13
3
1,865 Views
Last Modified: 2012-05-10
Hello,

I have back to back firewall topology and two ISP providers with 30 IPs per each.

ISP1: x.212.0.192/27
ISP2: x.189.188.64/27

I've configured each external network card with 4 IPs and set up ISP-R with Load-balancing. So my problem is that I can ping from internet only first IP on first external adapter when the second external adapter responds on all public IPs and vice-versa.

With further analysis I found the problem. Packets returns via wrong ethernet adapter.

First IP on first provider works correctly:
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.194 on eth0
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.194 on eth2
ICMP echo rply (60 bytes) from x.212.0.194 to x.255.234.4 on eth2
ICMP echo rply (60 bytes) from x.212.0.194 to x.255.234.4 on eth0

The second and others not:
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.195 on eth0
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.195 on eth2
ICMP echo rply (60 bytes) from x.212.0.195 to x.255.234.4 on eth3

while second provider:
ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.66 on eth1
ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.66 on eth3
ICMP echo rply (60 bytes) from x.189.188.66 to x.255.234.4 on eth3
ICMP echo rply (60 bytes) from x.189.188.66 to x.255.234.4 on eth1

ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.67 on eth1
ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.67 on eth3
ICMP echo rply (60 bytes) from x.189.188.67 to x.255.234.4 on eth3
ICMP echo rply (60 bytes) from x.189.188.67 to x.255.234.4 on eth1

I'm 100% sure that with linux and routing tables is everything OK because when I disable one public network adapter, connection works.

TMG IP configuration:
Ethernet adapter Internet SiOL:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : x.189.188.66
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.189.188.67
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.189.188.68
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.189.188.69
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   Default Gateway . . . . . . . . . : x.189.188.65

Ethernet adapter Internet T-2:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : x.212.0.194
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.212.0.195
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.212.0.196
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.212.0.197
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   Default Gateway . . . . . . . . . : x.212.0.193

Ethernet adapter Demilitarized Zone:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : 192.168.1.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : 172.16.0.1
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :

Active Routes:
Network Destination        Netmask          Gateway       Interface
          0.0.0.0          0.0.0.0     x.212.0.193     x.212.0.194
          0.0.0.0          0.0.0.0   x.189.188.65   x.189.188.66
     x.212.0.192  255.255.255.224         On-link      x.212.0.194
     x.212.0.194  255.255.255.255         On-link      x.212.0.194
     x.212.0.195  255.255.255.255         On-link      x.212.0.194
...
   x.189.188.64  255.255.255.224         On-link    x.189.188.66
   x.189.188.66  255.255.255.255         On-link    x.189.188.66
   x.189.188.67  255.255.255.255         On-link    x.189.188.66
...

Thanks for your time.
Matt
FW-Network.jpg
0
Comment
Question by:atrbanda
3 Comments
 
LVL 29

Expert Comment

by:pwindell
ID: 33672495
True redundancy is a fantasy,...it doesn't exist.
The dual links in the DMZ are pointless
The two WAN links outside the Linux is acting exactly like it should,...you are not going to get what you are looking for from that.
The onel true redundancy would come from having two line from the same ISP that come into the same routing device (a router, not a firewall).  The ISP then uses dynamic routing protocols (such as EIGRP) betweent eh routing device on your side of the pair of line and their routers on their side of the pair of lines.  It is 100% the job of the ISP,..they do it all, not you.
Example router config of a per-packet load balancing:
interface Serial0
 ip address 209.144.45.222 255.255.255.252
 ip load-sharing per-packet
 no ip route-cache cef
 load-interval 30
 no fair-queue
!
interface Serial1
 ip address 216.90.64.158 255.255.255.252
 ip load-sharing per-packet
 no ip route-cache cef
 load-interval 30
!
router eigrp 3
 network 209.16.209.0
 network 209.144.45.0
 network 216.90.64.0
 no auto-summary
 no eigrp log-neighbor-changes
It has to be layed out like this:
 

RedundantLines.jpg
0
 

Author Comment

by:atrbanda
ID: 33687742
I complicated here on the picture. Ok, forget it about front firewall. Let's say I have only TMG and two different ISP providers. I would like to achieve some loadbalancing (I know it is not good loadbalancer - balancing by sessions) and failover if one ISP provider fails. If I have only one public IP per ISP everything works fine (configured ISP-R). When I add additional IPs on each external adapter, routing on all IPs works only for "primary gateway".

Pinging First ISP:
IP1 Primary OK
IP2 OK
IP3 OK
...

Pinging Second ISP:
IP1 Primary OK
IP2 Fail
IP3 Fail
...
and vice-versa

For analysis I put Linux for front firewall just to see what's going out with traffic. As I wrote earlier, packets are returning via wrong network card:
Ping request for Second ISP IP2, returning via First ISP network adapter, package dropped.

My question is:
Has somebody such problem?
Is multi ip supported for ISP-R?
Is something wrong with my configuration or is this a bug in TMG?

Thanks.
Matt

FW1-Network.jpg
0
 
LVL 51

Accepted Solution

by:
Keith Alabaster earned 500 total points
ID: 33690831
you REALLY need to do some reading on the concepts of ISP -r for both a failover and a load-balanced scenario.

In resilience mode, only one connection is active - the second is to all intents and purposes shutdown. If you are able to ping them I believe you will find that traffic is gouing out of the first connection to the internet and then coming back to the second connection from OUTSIDE.

On the first connection, traffic will always leave on the primary address so I am not surprised you get no responses from the second, third or fourth addresses. I'll try it myself when I get home from work.

In load-balanced mode, both sessions are active simultaneously so you should get to IP 1 on both connections.

 
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article is in response to a question (http://www.experts-exchange.com/Networking/Network_Management/Network_Analysis/Q_28230497.html) here at Experts Exchange. The Original Poster (OP) requires a utility that will accept a list of IP addresses …
We recently endured a series of broadcast storms that caused our ISP to shut us down for brief periods of time. After going through a multitude of tests, we determined that the issue was related to Intel NIC drivers on some new HP desktop computers …
This Micro Tutorial will teach you how to censor certain areas of your screen. The example in this video will show a little boy's face being blurred. This will be demonstrated using Adobe Premiere Pro CS6.
As a trusted technology advisor to your customers you are likely getting the daily question of, ‘should I put this in the cloud?’ As customer demands for cloud services increases, companies will see a shift from traditional buying patterns to new…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

26 Experts available now in Live!

Get 1:1 Help Now