Solved

TMG ISP Redundancy with multiple IPs

Posted on 2010-09-13
3
1,861 Views
Last Modified: 2012-05-10
Hello,

I have back to back firewall topology and two ISP providers with 30 IPs per each.

ISP1: x.212.0.192/27
ISP2: x.189.188.64/27

I've configured each external network card with 4 IPs and set up ISP-R with Load-balancing. So my problem is that I can ping from internet only first IP on first external adapter when the second external adapter responds on all public IPs and vice-versa.

With further analysis I found the problem. Packets returns via wrong ethernet adapter.

First IP on first provider works correctly:
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.194 on eth0
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.194 on eth2
ICMP echo rply (60 bytes) from x.212.0.194 to x.255.234.4 on eth2
ICMP echo rply (60 bytes) from x.212.0.194 to x.255.234.4 on eth0

The second and others not:
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.195 on eth0
ICMP echo req (60 bytes) from x.255.234.4 to x.212.0.195 on eth2
ICMP echo rply (60 bytes) from x.212.0.195 to x.255.234.4 on eth3

while second provider:
ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.66 on eth1
ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.66 on eth3
ICMP echo rply (60 bytes) from x.189.188.66 to x.255.234.4 on eth3
ICMP echo rply (60 bytes) from x.189.188.66 to x.255.234.4 on eth1

ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.67 on eth1
ICMP echo req (60 bytes) from x.255.234.4 to x.189.188.67 on eth3
ICMP echo rply (60 bytes) from x.189.188.67 to x.255.234.4 on eth3
ICMP echo rply (60 bytes) from x.189.188.67 to x.255.234.4 on eth1

I'm 100% sure that with linux and routing tables is everything OK because when I disable one public network adapter, connection works.

TMG IP configuration:
Ethernet adapter Internet SiOL:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : x.189.188.66
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.189.188.67
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.189.188.68
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.189.188.69
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   Default Gateway . . . . . . . . . : x.189.188.65

Ethernet adapter Internet T-2:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : x.212.0.194
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.212.0.195
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.212.0.196
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   IPv4 Address. . . . . . . . . . . : x.212.0.197
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   Default Gateway . . . . . . . . . : x.212.0.193

Ethernet adapter Demilitarized Zone:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : 192.168.1.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . : 172.16.0.1
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :

Active Routes:
Network Destination        Netmask          Gateway       Interface
          0.0.0.0          0.0.0.0     x.212.0.193     x.212.0.194
          0.0.0.0          0.0.0.0   x.189.188.65   x.189.188.66
     x.212.0.192  255.255.255.224         On-link      x.212.0.194
     x.212.0.194  255.255.255.255         On-link      x.212.0.194
     x.212.0.195  255.255.255.255         On-link      x.212.0.194
...
   x.189.188.64  255.255.255.224         On-link    x.189.188.66
   x.189.188.66  255.255.255.255         On-link    x.189.188.66
   x.189.188.67  255.255.255.255         On-link    x.189.188.66
...

Thanks for your time.
Matt
FW-Network.jpg
0
Comment
Question by:atrbanda
3 Comments
 
LVL 29

Expert Comment

by:pwindell
ID: 33672495
True redundancy is a fantasy,...it doesn't exist.
The dual links in the DMZ are pointless
The two WAN links outside the Linux is acting exactly like it should,...you are not going to get what you are looking for from that.
The onel true redundancy would come from having two line from the same ISP that come into the same routing device (a router, not a firewall).  The ISP then uses dynamic routing protocols (such as EIGRP) betweent eh routing device on your side of the pair of line and their routers on their side of the pair of lines.  It is 100% the job of the ISP,..they do it all, not you.
Example router config of a per-packet load balancing:
interface Serial0
 ip address 209.144.45.222 255.255.255.252
 ip load-sharing per-packet
 no ip route-cache cef
 load-interval 30
 no fair-queue
!
interface Serial1
 ip address 216.90.64.158 255.255.255.252
 ip load-sharing per-packet
 no ip route-cache cef
 load-interval 30
!
router eigrp 3
 network 209.16.209.0
 network 209.144.45.0
 network 216.90.64.0
 no auto-summary
 no eigrp log-neighbor-changes
It has to be layed out like this:
 

RedundantLines.jpg
0
 

Author Comment

by:atrbanda
ID: 33687742
I complicated here on the picture. Ok, forget it about front firewall. Let's say I have only TMG and two different ISP providers. I would like to achieve some loadbalancing (I know it is not good loadbalancer - balancing by sessions) and failover if one ISP provider fails. If I have only one public IP per ISP everything works fine (configured ISP-R). When I add additional IPs on each external adapter, routing on all IPs works only for "primary gateway".

Pinging First ISP:
IP1 Primary OK
IP2 OK
IP3 OK
...

Pinging Second ISP:
IP1 Primary OK
IP2 Fail
IP3 Fail
...
and vice-versa

For analysis I put Linux for front firewall just to see what's going out with traffic. As I wrote earlier, packets are returning via wrong network card:
Ping request for Second ISP IP2, returning via First ISP network adapter, package dropped.

My question is:
Has somebody such problem?
Is multi ip supported for ISP-R?
Is something wrong with my configuration or is this a bug in TMG?

Thanks.
Matt

FW1-Network.jpg
0
 
LVL 51

Accepted Solution

by:
Keith Alabaster earned 500 total points
ID: 33690831
you REALLY need to do some reading on the concepts of ISP -r for both a failover and a load-balanced scenario.

In resilience mode, only one connection is active - the second is to all intents and purposes shutdown. If you are able to ping them I believe you will find that traffic is gouing out of the first connection to the internet and then coming back to the second connection from OUTSIDE.

On the first connection, traffic will always leave on the primary address so I am not surprised you get no responses from the second, third or fourth addresses. I'll try it myself when I get home from work.

In load-balanced mode, both sessions are active simultaneously so you should get to IP 1 on both connections.

 
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Greetings, Experts! First let me state that this website is top notch. I thoroughly enjoy the community that is shared here; those seeking help and those willing to sacrifice their time to help. It is fantastic. I am writing this article at th…
Many of us in IT utilize a combination of roaming profiles and folder redirection to ensure user information carries over from one workstation to another; in my environment, it was to enable virtualization without needing a separate desktop for each…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now