MPLS Internet breakout on Watchguard
Hi
I have a customer with four sites, connected via MPLS. Each site has internet through fibre connections seperate to the MPLS.
They are now going to be using the internet breakout on their MPLS and the fiber lines will be disconnected.
They have a seperate fiber line and bonded ADSL line to the MPLS at each site (one for redundancy)
The MPLS provider has there own routers at each site and these plug into our Watchguards (XTM 505) currently as trusted interfaces with all traffic allowed both ways.
In order to use load balancing with the Watchguard I want to change these to External interfaces, however when I do the internet works via the breakout but the LAN traffic between sites doesn't. I have a feeling that this is a NAT issue but cannot think where to set it - Any ideas how to get all traffic flowing this way. (the rule is still any any as the MPLS is providing protection via their firewall cluster)
I have it working as a pair of trusted interfaces with a metric of 1 on the Fiber and a metric of 10 on the ADSL, and all traffic is flowing, however when I pull the plug on the Fiber the traffic stops for everything?...(All of the static routes have been entered). Preferably I want the MPLS interfaces to be setup as External on the watchguard to allow true failover to work but at the moment only internet is working?......
Any Watchguard experts out there who can help?....
I have a customer with four sites, connected via MPLS. Each site has internet through fibre connections seperate to the MPLS.
They are now going to be using the internet breakout on their MPLS and the fiber lines will be disconnected.
They have a seperate fiber line and bonded ADSL line to the MPLS at each site (one for redundancy)
The MPLS provider has there own routers at each site and these plug into our Watchguards (XTM 505) currently as trusted interfaces with all traffic allowed both ways.
In order to use load balancing with the Watchguard I want to change these to External interfaces, however when I do the internet works via the breakout but the LAN traffic between sites doesn't. I have a feeling that this is a NAT issue but cannot think where to set it - Any ideas how to get all traffic flowing this way. (the rule is still any any as the MPLS is providing protection via their firewall cluster)
I have it working as a pair of trusted interfaces with a metric of 1 on the Fiber and a metric of 10 on the ADSL, and all traffic is flowing, however when I pull the plug on the Fiber the traffic stops for everything?...(All of the static routes have been entered). Preferably I want the MPLS interfaces to be setup as External on the watchguard to allow true failover to work but at the moment only internet is working?......
Any Watchguard experts out there who can help?....
Paul Solovyovsky
what do the logs on the WG show when it's not working?
the MPLS provider should be providing you with two interfaces, one for internal traffic and one for Internet.
If they are only providing you with a single interface, I would use a switch (or three ports in a VLAN on an existing switch) to connect to two external interfaces on the watchguard, one external interface would be the Internet connection running NAT and the other would be the MPLS connection in "transparent" mode with ACLs to allow traffic to and from the other sites.
If they are only providing you with a single interface, I would use a switch (or three ports in a VLAN on an existing switch) to connect to two external interfaces on the watchguard, one external interface would be the Internet connection running NAT and the other would be the MPLS connection in "transparent" mode with ACLs to allow traffic to and from the other sites.
ASKER
@ArneLovius
The setup of the MPLS is like below
Site 1
1.1.1.1 - 1.1.1.9 (MPLS Internet IP) >> 192.168.100.1 (MPLS Internal IP) >> 192.168.50.1-254 (Watchguard internal IP range)
At the moment the connection is transparent and the other sites in the MPLS can be accessed as can the internet. We only have a single interface assigned from the MPLS routers so the switch option would be the way forward.
Two things though,
1 - What IP goes on the external interface as this and the trusted interface for the internal traffic cannot be the same
2 - How do I NAT to my internal servers?
Thanks
The setup of the MPLS is like below
Site 1
1.1.1.1 - 1.1.1.9 (MPLS Internet IP) >> 192.168.100.1 (MPLS Internal IP) >> 192.168.50.1-254 (Watchguard internal IP range)
At the moment the connection is transparent and the other sites in the MPLS can be accessed as can the internet. We only have a single interface assigned from the MPLS routers so the switch option would be the way forward.
Two things though,
1 - What IP goes on the external interface as this and the trusted interface for the internal traffic cannot be the same
2 - How do I NAT to my internal servers?
Thanks
As a picture is supposedly worth a thousand words...
ASKER
Thanks for the pic, that does explain a lot.
However the WG won't let me setup two interfaces (one external, one trusted) that are on the same subnet. Should I set the external interface with the External IP range supplied by the MPLS provider?.....
However the WG won't let me setup two interfaces (one external, one trusted) that are on the same subnet. Should I set the external interface with the External IP range supplied by the MPLS provider?.....
Each site should have its own subnet and the MPLS "network" should be using a different subnet that does not overlap with any of the sites.
for example
Site A 192.168.1.0/24
Site B 192.168.2.0/24
Site C 192.168.3.0/24
MPLS 172.16.1.0/24
for example
Site A 192.168.1.0/24
Site B 192.168.2.0/24
Site C 192.168.3.0/24
MPLS 172.16.1.0/24
ASKER
That's correct, so using these IP's as an example I am assuming this is how it would be connected
Internal LAN IP for Site A 192.168.1.0/24
External WAN IP for Internet (172.16.1.10)
Trusted IP for MPLS connectivity to the other sites (172.16.1.11)
*Connected as per the diagram above
The trouble is it won't allow the External and Trusted IP's to be on the same subnet, so how do I set this correctly?.....
Internal LAN IP for Site A 192.168.1.0/24
External WAN IP for Internet (172.16.1.10)
Trusted IP for MPLS connectivity to the other sites (172.16.1.11)
*Connected as per the diagram above
The trouble is it won't allow the External and Trusted IP's to be on the same subnet, so how do I set this correctly?.....
I don't follow what you mean by "Trusted IP for MPLS...."
ASKER
The engineer that set this all up originally set the MPLS as a trusted interface on the watchguard, allowing any<>any to allow internal traffic between each site. Internet was via another fiber connection setup as an external interface.
Now they have internet breakout on the MPLS and the separate fiber line is going to be disconnected.
I have only recently taken in this site and I have never dealt with MPLS before.
When I say Trusted IP for MPLS I mean the current "trusted interface for MPLS"
I really need some help on setting this up correctly, expecially in regards to
The IP addressing of the interfaces so traffic flows between sites
Secure access to the internet via the breakout
.......Thanks
Now they have internet breakout on the MPLS and the separate fiber line is going to be disconnected.
I have only recently taken in this site and I have never dealt with MPLS before.
When I say Trusted IP for MPLS I mean the current "trusted interface for MPLS"
I really need some help on setting this up correctly, expecially in regards to
The IP addressing of the interfaces so traffic flows between sites
Secure access to the internet via the breakout
.......Thanks
so leave the MPLS as is, and just create a new interface for the Internet connection, or if an Internet connection is already in place, just change the addressing
ASKER
.....What would the addressing be on the internet interface?
I can't have two interfaces on the same subnet
When I set the MPLS as NAT the internet works but the remote site connectivity doesn't....
I can't have two interfaces on the same subnet
When I set the MPLS as NAT the internet works but the remote site connectivity doesn't....
a diagram of where you have what addresses might be useful
ASKER
I have attached an overview of the network and the interface config for Site A
They currently use a fiber line as their internet access and this is the external interface marked on the capture. This is going to be decommissioned at the end of the month.
This line has 14 IP's associated with it for email and internal web servers etc, which we will need to change the A records to the internet breakout IP range on the MPLS.
When the MPLS interface (the one highlighted as the trusted interface) is set as it is then both the internet and the other sites can be reached. However I cannot use any NAT rules to map the traffic to my internal servers?....
When I change it to external I can set the NAT rules, and the internet works but I cannot see the other sites?....
I could use the switch idea for two interfaces to the Watchguard as you suggested but I would need a different subnet on each interface..
It's tricky to explain but if you need any more info let me know
*The IP ranges are made up for the purposes of the question
Thanks
Network.jpg
Site-A-interface-config.jpg
They currently use a fiber line as their internet access and this is the external interface marked on the capture. This is going to be decommissioned at the end of the month.
This line has 14 IP's associated with it for email and internal web servers etc, which we will need to change the A records to the internet breakout IP range on the MPLS.
When the MPLS interface (the one highlighted as the trusted interface) is set as it is then both the internet and the other sites can be reached. However I cannot use any NAT rules to map the traffic to my internal servers?....
When I change it to external I can set the NAT rules, and the internet works but I cannot see the other sites?....
I could use the switch idea for two interfaces to the Watchguard as you suggested but I would need a different subnet on each interface..
It's tricky to explain but if you need any more info let me know
*The IP ranges are made up for the purposes of the question
Thanks
Network.jpg
Site-A-interface-config.jpg
leave the existing config as is
add another external interface with the IP address of the Internet connection provided on the MPLS router
as per my previous, use a switch, or configure a L2 VLAN on a managed switch.
Unplug the network cable from the MPLS router and connect it to the switch
Add a new network cable from the switch to the MPLS router
Add a new network cable from the switch to the "new" external interface
It should look something like this
add another external interface with the IP address of the Internet connection provided on the MPLS router
as per my previous, use a switch, or configure a L2 VLAN on a managed switch.
Unplug the network cable from the MPLS router and connect it to the switch
Add a new network cable from the switch to the MPLS router
Add a new network cable from the switch to the "new" external interface
It should look something like this
ASKER
Ok -I fully understand your explanation and it all makes sense, however the issue I have is what the IP's should be on the two interfaces, from your example....
New External Interface = ?.?.?.?
MPLS Firewall Interface = 192.168.0.2 /29
- When I test the interface set as "New External Interface" with the IP 192.168.0.2 then the internet works and you can NAT through to the internal network (how I can map multiple external IP's is still a mystery).
However I can't use that IP or subnet on both interfaces, the WG doesn't allow it and I don't think router/firewall would.
...So, I try setting the "New External Interface" as 3.1.1.1/28 which is the external IP range of the MPLS internet breakout and that doesn't work.....and I don't see how it can be.
Thanks for your help so far, I know what I want to get the setup to do but I just cannot get it to work
New External Interface = ?.?.?.?
MPLS Firewall Interface = 192.168.0.2 /29
- When I test the interface set as "New External Interface" with the IP 192.168.0.2 then the internet works and you can NAT through to the internal network (how I can map multiple external IP's is still a mystery).
However I can't use that IP or subnet on both interfaces, the WG doesn't allow it and I don't think router/firewall would.
...So, I try setting the "New External Interface" as 3.1.1.1/28 which is the external IP range of the MPLS internet breakout and that doesn't work.....and I don't see how it can be.
Thanks for your help so far, I know what I want to get the setup to do but I just cannot get it to work
"new external" would be one of the public IP address provided by the MPLS provider
you would then add the rest of the addresses to the watchguard for your other NAT/PAT rules.
you would then add the rest of the addresses to the watchguard for your other NAT/PAT rules.
ASKER
Well this is what I thought it should be, however when I set it with the range they have given us it doesn't work, the tracert dies as soon as it leaves the watchguard.
Wouldn't the external IP's on the MPLS be a couple of hops away on the MPLS network?...
Wouldn't the external IP's on the MPLS be a couple of hops away on the MPLS network?...
you would also need to change the routing on the watchguard
I'm not sure I follow you, it is not possible to say how many hops something would be on an MPLS network, the internet "breakout" could be the next logical hop...
I'm not sure I follow you, it is not possible to say how many hops something would be on an MPLS network, the internet "breakout" could be the next logical hop...
ASKER
How would I change the routing?
At the moment there are static routes directing traffic to the remote sites through the MPLS, I assume these will stay....
At the moment there are static routes directing traffic to the remote sites through the MPLS, I assume these will stay....
they would stay, but the route to 0.0.0.0 0.0.0.0 would need to change
ASKER
The default route exists on the MPLS already. Once the external interface is set on the watchguard then the internet traffic will go via that, and the current routes will direct the traffic for the other sites through the MPLS connection.
Before I go to the MPLS provider and question their setup I need to know I am getting this right....
So on the external interface on the WG this should be set as the external IP range from the MPLS provider? If this is the case I need to talk to them as testing as shown it doesn't work.
Then the transparent connection that I use at the moment remains and traffic to the remote sites should carry on working.
Once I have the ability to NAT the internet traffic then I can adjust the rules as needed...
Before I go to the MPLS provider and question their setup I need to know I am getting this right....
So on the external interface on the WG this should be set as the external IP range from the MPLS provider? If this is the case I need to talk to them as testing as shown it doesn't work.
Then the transparent connection that I use at the moment remains and traffic to the remote sites should carry on working.
Once I have the ability to NAT the internet traffic then I can adjust the rules as needed...
There is the MPLS private connection and the MPLS Internet connection, can you please be specific.
The site to site link will need to have static routes to the remote sites
The Internet link should have a static route to 0.0.0.0 0.0.0.0 (default gateway)
Unless, their design is that they provide a "managed firewall" so all traffic goes to the MPLS "cloud" and they then manage your Internet breakout, something like the below
The site to site link will need to have static routes to the remote sites
The Internet link should have a static route to 0.0.0.0 0.0.0.0 (default gateway)
Unless, their design is that they provide a "managed firewall" so all traffic goes to the MPLS "cloud" and they then manage your Internet breakout, something like the below
ASKER
Yeah....they do manage the internet breakout.....as you outline above
Am I screwed then when it comes to setting up the interfaces as I want them?...
Am I screwed then when it comes to setting up the interfaces as I want them?...
if they are managing the internet breakout, then the watchguard is mostly redundant, you can use it as a router, but you need to need to allow all traffic through it....
ASKER
But how can I NAT the traffic from the managed internet which they are directing to my watchguard to the right internal servers?
I think we are at cross purposes
If you have Internet connectivity, then they should have provided you with a routed "block" of IP addresses.
If they are providing a managed firewall service, then they would manage NAT etc
If you have Internet connectivity, then they should have provided you with a routed "block" of IP addresses.
If they are providing a managed firewall service, then they would manage NAT etc
ASKER
Possibly...
The MPLS provider requested a firewall policy sheet on which we wrote down all of the NAT rules we wanted putting in. As the WG was remaining I assumed that their internal endpoint would be the MPLS interface on the WG at which point the traffic would be natted at the watchguard through to the internal servers - double NATTING effectively.
I guess this isn't going to work
I also guess that life would be a lot easier if the WG was out of the mix......which the Company don't want to do
Any suggestions how this can be rectified?......
I can let all traffic through on MPLS so internet and remote site access works its just the NAT bit that is doing my head in....
The MPLS provider requested a firewall policy sheet on which we wrote down all of the NAT rules we wanted putting in. As the WG was remaining I assumed that their internal endpoint would be the MPLS interface on the WG at which point the traffic would be natted at the watchguard through to the internal servers - double NATTING effectively.
I guess this isn't going to work
I also guess that life would be a lot easier if the WG was out of the mix......which the Company don't want to do
Any suggestions how this can be rectified?......
I can let all traffic through on MPLS so internet and remote site access works its just the NAT bit that is doing my head in....
You could do double NAT, but I would advise against it.
ASKER
Is the solution to remove the Watchguards?
or can the NAT rules from the MPLS provider be directed straight through to the internal network?
I really need some advice here......how do Companies deal with this normally?.....
or can the NAT rules from the MPLS provider be directed straight through to the internal network?
I really need some advice here......how do Companies deal with this normally?.....
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Hi ArneLovius
I was on site testing this last night and solved the issue, but still can't quite get my head round how
Basically I set the interface to the MPLS up as an "External" interface and I was having the same issue where the internet and external NAT to internal servers would work but remote connections to the other sites wouldn't.
As the MPLS interface has the IP 192.168.x.x/29 I removed the Dynamic NAT setting that is installed by default on the Watchguard which makes anything with the range
192.168.0.0/16 > Any-External
Removing this allowed the traffic to work between remote sites, I can NAT traffic on the Watchguard from the MPLS through to Internal servers based on port numbers etc and I can get the internet.
Obviously some sort of NAT issue and both sides of the network having a 192.168.0.0 subnet but can you explain (maybe with one of your pictures:-) how this has allowed it to work?.......
I was on site testing this last night and solved the issue, but still can't quite get my head round how
Basically I set the interface to the MPLS up as an "External" interface and I was having the same issue where the internet and external NAT to internal servers would work but remote connections to the other sites wouldn't.
As the MPLS interface has the IP 192.168.x.x/29 I removed the Dynamic NAT setting that is installed by default on the Watchguard which makes anything with the range
192.168.0.0/16 > Any-External
Removing this allowed the traffic to work between remote sites, I can NAT traffic on the Watchguard from the MPLS through to Internal servers based on port numbers etc and I can get the internet.
Obviously some sort of NAT issue and both sides of the network having a 192.168.0.0 subnet but can you explain (maybe with one of your pictures:-) how this has allowed it to work?.......
I would guess that the interface that it was using for NAT wasn't the correct interface for the MPLS provided Internet access.
ASKER
Thanks for sticking with me on this one, your help was much appreciated