Solved

Layer 2 Vs Layer 3 across multiple sites

Posted on 2010-08-25
19
1,365 Views
Last Modified: 2012-05-10
I have 2 sites connected via a dedicated 100Mbps line.  One site has a 6509 and the other a 4507 (Both layer 3 switches).  I could just trunk the vlans accross the two switches but don't want broadcast across that 100Meg connection.  So I should route, but routing is less efficient than switching.  So I am not sure which direction to go.  I am personally leaning toward routing but I have never designed a multisite network and want to know what is best practice in this senario?  Thanks for the feedback.
0
Comment
Question by:MorrisJay
  • 8
  • 6
  • 4
  • +1
19 Comments
 
LVL 2

Expert Comment

by:fs40490
Comment Utility
Really comes down to your requirements.  If there is not a requirement that devices at both sites need to be on the same layer 2 network (so VLANs actually shared across the sites), then just route.  There is not a huge overhead on the routing in your scenario and with the devices you are using.  

Again though it really depends on you requirements of the solution.
0
 
LVL 3

Expert Comment

by:petelettin
Comment Utility
I believe most cisco layer 3 switches route the first packet and switch the rest.

We have a similar setup which was configured by a third party consultant and they used routing.

I think we would change to a switch trunk if given the chance.


Pete :-)
0
 

Author Comment

by:MorrisJay
Comment Utility
fs40490 - No real requirment.  We had servers on the same vlan that spanned both devices but are in the process of re-IPing our servers at the HQ (the 6509).  The only servers really on the 6509 side are DC's, DHCP, and Print servers.  The 4507 is at a Co-Location and has a couple DC's, our mail servers, other production boxes and our exit to the internet.

Pete-  Why would you trunk if given the chance?
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
AD and Exchange2007 & 2010 use AD Sites and Services for handling replication and Mail Routing Groups over the WAN links.  AD Sites and Services, by design, requires each site be a different subnet.

Running the same Broadcast Domain over a WAN link causes all broadcasts to go over the WAN links which already stressed due to the fact that they are low bandwidth compared to the rest of the LAN.
0
 
LVL 3

Expert Comment

by:petelettin
Comment Utility
Why,
easier setup of QoS for voip
more secure
easier to setup a layer 2 tunnel from remote site to firewall without routing through remote and local layer 3 switch cores.

Pete :-)
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
QoS for VoIP doesn't mean anything if you don't have VoIP.  Not everyone that has VoIP even needs to use QoS.
Certainly not more secure since you loose the ACLs (and possible NAT) that the L3 Router can provide.
Routing through Layer3 Switches is not  something "hard" that would require doing something  else instead of it.  Active Directory Sites and Services uses Layer3 as a boundary in order to function.   A Layer2 Tunnel would have to be defined in the context you mean it and why you need it or want it at all..
0
 
LVL 3

Accepted Solution

by:
petelettin earned 250 total points
Comment Utility
pwindell,

Voip works without QoS but users will suffer jitter and delay if network is busy. (We do) QoS gives voip priority over data e.g.file transfers.

ACLs can be appliend at the vlan on layer 2,  we do already.
Routed over layer 3 all traffic across link is on same subnet.

But we currently already tunnel a remote office (of another company) over the layer 3 network and break it out at the main site over a seperate internet connection.
It has to be routed hitting 3 extra devices and cannot be sent over layer 2 vlan direct to the internet router.

AD is not an issue as remote network is completely seperate and only used for internet services.

Ther are obviously many different scenarios where 1 would have advantage over the other, in our case I think layer 2 trunk would have been a better choice.

We have over 400 switches in stacks of approx 6 switches, over 6000 outlets and over 700 vlans.
Our WAN link is a 2 x 100mb Radio.

Our layer 2 would have a layer 3 core at each site (as now) which would allow for seperate subnets at remote site allowing AD sites an Services to function in the same way.
 
It's much easier to draw the logical network.

Why do I feel the need to defend my statement. It is after all my expert opinion based on my own experience of 20+ years.

Pete :-)
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
Right, that's true, but we are dealing with the person's network who asked the question,..not your network.
BTW - we use VoIP without QoS.  Works just fine,..even with sharing the segment with the PC.  It just depends on the conditions.  We are currently building a separate physical network for them due to other problems not related to QoS.
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
We also need the solution to be tied to the skill level of the person who has to work with it everyday.  The simple straight forward method of each Site being it's own L3 subnet and then just route between the Sites in a straight forward way will be the easiest to deal with.  If he thinks he wants something more complex, and thinks he can handle it, then fine, he can say so.
We don't know what his AD design is and this normal L3 method is neutral to the AD design.
   We don't know that he even has an VoIP Phones,...heck I wish we never had them, ours suck,...I'd almost kill to go back to the old non-IP based system we had before, they did as much as these VoIP phones we got stuck with do and they actually worked.
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 

Author Comment

by:MorrisJay
Comment Utility
We are currently using VoIP with no QoS.  At this point it is not an issue however I can foresee a problem in the future as we have what has been described to me as an "Open Culture" here which means that people are free to do what they want on their work computers.  We have people streaming Pandora and the likes.  In fact we have one user who chews threw about 4GB - 7GB of internet data a day.  So I want to keep QoS an easy and viable option for the future.  I don't want to have to come back in 3 years and redesign our network.  That being said I don't see why QoS is anymore difficult routing than switching but again I am a green when it comes to that.  What makes routing with QoS more difficult than trunking?

pwindell - I appreciate your insight however I don't really see what sites and services has to do with it.  Even if we trunked across our WAN we are still routing the vlans in the layer 3 switches.
0
 
LVL 29

Assisted Solution

by:pwindell
pwindell earned 250 total points
Comment Utility
Hi, Morris Jay
Sorry the thread is getting stretched out so long.
redesign our network.  That being said I don't see why QoS is anymore difficult routing than switching but again I am a green when it comes to that.  What makes routing with QoS more difficult than trunking?
QoS does not change the bandwidth over a particular Line.  QoS prioritizes certain types of packets -vs- certain other types of packets,...but the total bandwidth stays the same.  So QoS or not QoS doesn't really change the design. So if you need QoS later that is fine.  We (as you) are running VoIP without QoS and the sound quality is fine (it's about the only thing our stupid phones do right).

pwindell - I appreciate your insight however I don't really see what sites and services has to do with it.  Even if we trunked across our WAN we are still routing the vlans in the layer 3 switches.
Well,..that is really two questions that needs two answers.  
1. The AD Domain design and the Exchange Organization desing dictates if the use of Sites and Services is needed.  If you have a single Forest that spans WAN links then you need a different subnet on each side of the WAN link.  The proper design would mean that you have a DC at each location.  The Sites & Servers (which use the subnets as a boundary and an identifier) to control Replication over the WAN link and help to insure that users log into the closest "lowest cost" DC when they authenticate so that they do not add additional stress to the slower WAN Links.  Hence you want each location to be a separate subnet,...do not span a single subnet over a WAN link.  Also keep in mind that Ehternet as a technology looses efficiency at or around 250-300 Hosts on an IP segment,...depending on the size of the sites,...combining two on more into a single segment and bump it over that many Hosts. This would already be bad on a 100mbps LAN, but throw in a slow WAN link in the middle of the subnet and it gets much worse.
2. Trunking.  Trunking is running multiple VLANs over the same wire.  If you have each location in its own subnet so that you do not run multiple IP# segments over the same wire,... then there is no trunking,..so it becomes a non-issue.     Routing,...well, routing is routing.  Routing does not create this big overhead that I think you believe it does,...in fact not using it is what creates the problems.  I agree with what fs40490 was trying to say in the first reply you received.  The purpose of L3 Routing is to make traffic more efficient,...not less efficient,...it breaks up broadcast domains into smaller more efficient pieces to reduce the load on the "wire".   Most of your traffic is going to stay within each site,...most of the traffic is not going to be going between the sites,...hence most of the traffic is not going to be precessed by the Router.
 
0
 
LVL 3

Expert Comment

by:petelettin
Comment Utility
If you look at my original comment I said routing was the option.

There is no detail of what the author requires over 2 sites with layer 3 switch at each end until after my first post.

I made a 1 line comment about my own experience and was then asked why.
YOU then analysed my response. Which was totally off the subject of the original question! as we're supposed to be helping him not attacking me!

And BTW our voip does suffer jitter and low quality across the link although the users don't seem to notice enough to complain. the rest of the network has QoS and is much clearer.

On a small network the router chatter will probably be more than the broadcast traffic especially with Ciscos.

re the voip, I think the poster will know if he uses it or not and if this affects his situation.

I have 4 more WAN links all of which are routed for the same reason as in the first post by fs40490

Pete :-)
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
No one is attacking you, Pete, and I agreed with your first post, I just didn't agree with the second one.  
0
 
LVL 3

Expert Comment

by:petelettin
Comment Utility
Ok, peerhaps I'm being a bit touchy, well my second post was in response to a question and I have all sorts of unusual reasons for my thinking my link may be better trunked.

In a more "simple" network design routing is the way to go.

your points about AD are valid pwindell, we use Sites and Services and have remote ADCs in case the link goes down etc.

Routing is only very slightly more difficult to setup on the L3 switch as is the ip addressing on the network but you already have this covered with a dhcp server etc.

Pete :-)
0
 

Author Comment

by:MorrisJay
Comment Utility
I really appriciate both your guys' insight.  Unfortunently I get and agree with both of your points which puts me right back to where I was in the begining.  I knew that both layer 2 and layer 3 are viable options but didn't know if there was a "standard" way of working with layer 3 switches over a dedicated WAN link, which apparently there is not.  I will split the points.

Again thank you for your insight.
0
 

Author Closing Comment

by:MorrisJay
Comment Utility
I get and agree with both of your points which puts me right back to where I was in the begining.  I knew that both layer 2 and layer 3 are viable options but didn't know if there was a "standard" way of working with layer 3 switches over a dedicated WAN link, which apparently there is not.
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
The trunking is probably the main thing I would try to avoid.  Trunking is a move away from efficiency, not towards it.  I think you (Pete) are better off the way you already have it.  Trunking is a move toward convenience at the expense of  efficiency.  For example, if you have two subnets at one particular site and want to "span" them across to a second site over a WAN link.  The efficient way would be to run two WAN links between the two sites and bridge each so each subnet has its own separate link.  but that is not financially convenient and no one does that,...so you trunk two subnets over the same WAN link and sacrifice performance for convenience .
The same situation sometimes exists on the backbone links between L2 switches (which makes them Trunk Lines), but that is why those links usually run 10x faster than the rest of the switch ports.  Unfortunately you can't make the WAN link run 10x faster than the regular LAN cabling.
0
 
LVL 29

Expert Comment

by:pwindell
Comment Utility
I think fs40490 gave the best answer and gave it first, and was short, simple, and to the point.  I think he should have gotten credit.  I don't need the points that bad.
0
 
LVL 3

Expert Comment

by:petelettin
Comment Utility
I agree fs40490 should have got the points. I'm not here for that reason, only to try and be helpful.
 
Pete :-)
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Quality of Service (QoS) options are nearly endless when it comes to networks today. This article is merely one example of how it can be handled in a hub-n-spoke design using a 3-tier configuration.
Getting hacked is no longer a matter or "if you get hacked" — the 2016 cyber threat landscape is now titled "when you get hacked." When it happens — will you be proactive, or reactive?
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now