We help IT Professionals succeed at work.

Routing between two networks across fiber

cstoneit1 asked
I have fiber connection that routes traffic across it being implemented. My main site where all data is going to be store is 192.168.1.x. I would like the second location to be 192.168.2.x. Here are my questions.
1. Should I use a router at both ends to plug into the fiber cisco switches from Charter or layer three switches?
2. Do I just leave them all on the same subnet and all the second location to just pull IP's from the main location.
3. If I need seperate equipment like cisco routers is there a simple config to load onto them?
Thank you.
Watch Question

John MeggersNetwork Architect

I would use routers, but I suppose as long as the proper physical connections and OS requirements (routing protocols, etc.) are supported you could use layer 3 switches.  

You will need another subnet for the link in between the sites.  Generally it will look like:

192.168.1.x --- router A --- <third subnet> --- router B ---  192.168.2.x

How you handle routing will depend on what your ISP is providing for you.  If it's a point-to-point circuit with addresses on either end that are in the same subnet, then you can use static routes. Router A will need a static route that tells it where is located, and Router B will need a static route telling it where is located.  (Or each could have a default route pointing towards the other.)

The more complex scenario is you're getting an MPLS link, in which case most people use BGP to exchange routes.  It's also possible you might be able to use a GRE tunnel between the sites to pass routes.  You'll probably have to jump off that bridge when you get there.  But it's important to know what's going to be required before you decide what equipment to purchase.  Last thing you want to do is purchase a couple of low-end L3 switches that don't support BGP, only to find out later on that's what you'll have to use.

jmeggers has given you a good answer.  I'm doing it with one router at each site to interface with the fiber link.
And, I use an "interim" subnet as shown.  And, you need the routes as shown:

Here's an example" fiber router "A" on local subnet "A" fiber router "A" on interim (fiber) subnet fiber router "B" on interim (fiber) subnet fiber router "B" on local subnet "B"

Some things to know:

The gateway for subnet "B" on router "A" is
The gateway for subnet "A" on router "B" is

On subnet "A", there is likely an internet gateway say:
Presumably, packets destined for subnet "B" from computers on subnet "A" will hit this gateway first.
So, there needs to be a route in the local gateway that points to the fiber gateway: >
and similarly on subnet "B".

Also note that packets arriving over the fiber *don't* hit the local internet gateway at all.  So, if the internet gateway device has stateful packet inspection working on the LAN side, it may well stop the return packets as there is no correspoinding "state".  This has to be turned off or modified somehow or you won't be successful.
And, this can be extended to a larger number of sites with one router per site and multiple routes - one for each site that's remote to the local.
If you are using "dark" fiber, then there is no difference to you between Layer 3 Switch and a Router (Routers handle WAN technologies better like MPLS, Frame Relay, T1, etc..)

If you use Layer 3 switch, configure the fiber ports as a router port with an IP address like jmeggers recommends.

You could also make the 2nd site an extension on the main site and use a Layer 2 connection between them. No need for routers and layer 3 switches. This  option would save money and work well for most cases. Plus the config is simple.

If you have over 100 computers at each site with little communication between them, I would recommend a Layer 3


There are 35 Computers at the Main Site Site A and 12 Computers at Site B. I was told leaving them all on the same LAN by just making the second site an extension is not a good practice, is this incorrect. If I did this I would just be able to plug both Cisco Fiber switches from Charter into Layer 2 switches and run with it correct?
If I were to use Layer three switches would HP 2600's work? If so would you be able to help with a config or would you recommend getting Cisco routers? I
A total of 47 computers on the same subnet is OK.

You can maybe use Layer 2 switches and one subnet for the  whole thing.
But then, are you going to use DHCP at all?
But then, do you have one or two internet gateways?  i.e. one for all at one site or one at each site?
If two internet gateways then you'd likely want two different DHCP leases with different gateways.  With 12 computers you can afford to NOT use DHCP.  But what about wireless laptops?
etc. etc.

I'm not suggesting an approach, just that you need to think about these things.
This is where networking is more art than science.

I'm working at a site where we have a Layer 2 LAN extended over 5 miles over dark fiber.

I'd go with layer 2 and upgrade to Layer 3 if it doesn't work well.

On DHCP over layer 2, Use long lease times in case of fiber loss.
Use the same vendor at both ends. Fiber connections can be 'touchy'

I personally don't like HP Pro Curves. Had a fiber speed problem. Auto speed/duplex didn't work.