Avatar of Techrunner
Techrunner
 asked on

10Giga Ethernet Link Between Two Buildings

Hello Experts,

We have two buildings 1KM away from each other.

At Building 1, we have 2 6500 Switches in existence.

At Building 2, we are going to install the switches such as follows

Ground Floor - 1 x WS-C3750X-24S-E
                        2 x WS C3560X- 48S-P
First Floor        2x WS-C3560X-48T-L

Second Floor   2xWS-C3560X-48T-L

All these switches will be connected to Cisco Fiber Switch ( Ground Floor)

Fiber Switch will be connected to Main Data Center Fiber 6500 Switch.
These switches will be in separate VTP domain.

I need the help
- How to 10 Giga Speed between Fiber Switch and 6500
- Selection of Fiber Optic Cable which is installed through underground
-  Fiber Gbic

Moreover, I dont vlans from 6500 to get copy on Building 2 Switch.


Thanks in advance
RoutersSwitches / HubsNetworking Hardware-Other

Avatar of undefined
Last Comment
Techrunner

8/22/2022 - Mon
ASKER CERTIFIED SOLUTION
Jody Lemoine

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
Hello Sir,

Thanks for your response.

- How I can find out whether which supervisor is running on 6500
- WS-C3750X-24S-E has 24 fiber port, but I am not sure what is there speed.

Thanks
Jody Lemoine

You can do a "show module" on the Catalyst 6500 to get the details on all installed modules, including the supervisors.

The 3750X's on-board ports are going to be 1Gb, regardless of whether it's a copper or fibre-based switch. That's why the C3KX-NM-10G module is necessary. It plugs into the expansion slot on the right-hand side of the unit and adds 10Gb SFP+ capability to the switch. Once that's installed, you can put an SFP-10G-LR SFP+ module into it in order to connect to your fibre.
Techrunner

ASKER
Just curious to know is will this be good lan design.

lan design
Result of Sh Module


sh module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL10425VXM
  2   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL10414EL3
  3   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     SAL1106GFFU
  5    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       SAL1109JJMR

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0019.2f57.1b74 to 0019.2f57.1b8b   2.5   12.2(14r)S5  12.2(18)SXD7 Ok
  2  0018.baaf.d9b8 to 0018.baaf.d9cf   2.5   12.2(14r)S5  12.2(18)SXD7 Ok
  3  001b.2a62.183c to 001b.2a62.186b  11.1   7.2(1)       8.3(0.156)RO Ok
  5  0016.46f9.2150 to 0016.46f9.2153   5.3   8.4(2)       12.2(18)SXD7 Ok

Mod Sub-Module                  Model              Serial        Hw     Status
--- --------------------------- ------------------ ------------ ------- -------
  1 Centralized Forwarding Card WS-F6700-CFC       SAL1107GYNF   2.1    Ok
  2 Centralized Forwarding Card WS-F6700-CFC       SAL1107GRJM   2.1    Ok
  5 Policy Feature Card 3       WS-F6K-PFC3B       SAL1102DRWS   2.3    Ok
  5 MSFC3 Daughterboard         WS-SUP720          SAL1109JM3P   2.6    Ok

Mod Online Diag Status
--- -------------------
  1 Pass
  2 Pass
  3 Pass
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
Jody Lemoine

The Supervisor Engine 720 3B doesn't have the 10Gb uplinks, unfortunately. You'll need to either upgrade to the VS-S720-10G-3C (or VS-S720-10G-3CXL) or add a 10Gb module (mentioned previously) to have 10Gb connectivity to the switch. Adding the module is the less expensive option, but if you're considering a supervisor upgrade anyway then moving to one that supports the uplinks you need makes sense.
Techrunner

ASKER
Sir,
Just to confirm is this the part number

SFP-10G-LR SFP+

How about LAN design ? Is it fine performance and redundancy wise
- Any specs for fiber cable to be installed under the ground ?
SOLUTION
Jody Lemoine

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
Sir,

Just a conclusion for the up links connectivity

Cisco 6500   - Get only SFP-10G-LR module
Cisco 3750  -  Get only SFP-10G-LR module
Fiber Cable  - Single Mode G.652

Right Sir ?
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
SOLUTION
Jody Lemoine

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
SOLUTION
Craig Beck

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
Thank you Sir

As per the design is concerned.
- We have numerous vlans on B1 Core Switches,
- For Building 2 I want to create separate vlans for them and default to B1 Cores
- I don't want B1 vlans to pass through or copy to B2 Switch.
Craig Beck

So use VTP transparent on all your switches, and prune all VLANs which don't need to be trunked to each switch stack.

For example, you want VLANs 101-200 in B1, VLANs 201-300 in B2 and VLANs 301-400 in B3.

At the core, create all VLANs.

On the trunk link to B1 you do:

interface PortChannel1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 101-200
 switchport trunk native vlan 50


but on the trunk to B2 you do:

interface PortChannel2
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 201-300
 switchport trunk native vlan 50


...and so on.

Make sense?
Jody Lemoine

If you want to be absolutely sure that this doesn't happen, you can set the 10Gb fibre interface on the 6500 as a routed interface. That way, no VLANs will pass at all. If you're going to do this, it makes more sense to configure the 3750X to be the default gateway for the building 2 VLANs and set the 10Gb fibre interfaces to be routed on both ends. Any traffic (if any) passing between the VLANs in building 2 can be handled locally by the 3750X and only traffic destined for the other building will be passed across the 10Gb fibre.
Experts Exchange has (a) saved my job multiple times, (b) saved me hours, days, and even weeks of work, and often (c) makes me look like a superhero! This place is MAGIC!
Walt Forbes
Jody Lemoine

On the 3750X, you can do:

interface TenGigabitEthernet1/0/0
 no switchport
 ip address 10.10.10.1 255.255.255.252

On the 6500, you would do:

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.2 255.255.255.252

(Interfaces and IPv4 addresses can be adjusted according to hardware configuration and addressing preference, of course.)

You can then define VLAN interfaces on the 3750X and put the default gateways for your VLANs there:

ip routing
!
interface VLAN50
 ip address 10.10.50.1 255.255.255.0
!
interface VLAN60
 ip address 10.10.60.1 255.255.255.0

The 3750X will be in control of your building 2 routing and will not leak VLANs to building 1, regardless of the VTP configuration on your switches in either building.

Then you can route between buildings using dynamic or static routing, and be absolutely sure that the VLANs in each of your buildings are separate.
Techrunner

ASKER
I read my mind. This is what I was thinking.

Can I also do in this way

B1 Core 1

interface vlan 50
ip address 10.10.50.1 255.255.255.0

interface Gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 101-200
 switchport trunk native vlan 50

but on the trunk to B2 you do:


interface vlan 50
ip address 10.10.50.2 255.255.255.0


interface gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 201-300
 switchport trunk native vlan 50
Jody Lemoine

This will work too, but it requires that you take steps to isolate your VTP configurations in each building, either by making all switches transparent or by setting different domains and passwords. It will also mean that all inter-VLAN routing between VLANs in building 2 will traverse your fibre to building 1.
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
I want to keep inter vlan routing for B2 the local switch itself

Can I achieve this with HSRP becoz we have this

On the 6500,

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.2 255.255.255.252
Jody Lemoine

If you want to keep inter-VLAN routing for B2 on the local switch itself, then the configuration needs to be on the 3750X and not on the 6500, so there's no need for HSRP there.

If you want inter-VLAN routing for B2 to be on the remote switch, then you would go with craigbeck's suggestion of passive VLANs across the 10Gb fibre and put your HSRP (if needed) on the VLAN interfaces on the 6500.
Techrunner

ASKER
I am thinking to configure this and keep inter vlan routing on local switch.

However I have redundancy at B1, If core 1 goes down, how B2 Switch will route the traffic to Core2 without any manual intervention.
I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck
Techrunner

ASKER
I am thinking to configure this and keep inter vlan routing on local switch.

However I have redundancy at B1, If core 1 goes down, how B2 Switch will route the traffic to Core2 without any manual intervention.
SOLUTION
Jody Lemoine

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
I guess I should I  go with this Thanks

Just one 1 more query, I am going to place 15 U Wall Mounted Cabinet on each floor

Ground Floor Will hold 3 Switch

First floor will hold 2 x 48
Second floor will hold 2 x 48

Room Space Width 1m x Depth 1.5 m

You would like to shed some light, if thats fine.
Jody Lemoine

None of those switches are more than a half metre from front to back, so that should be more than adequate.
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
Thank You Sir,

I will just make final plan if there is anything I'll update you.
Craig Beck

Agree with jodylemoine - L3 distribution layer is the way to go.
Techrunner

ASKER
Thanks craigbeck
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
Craig Beck

You may be able to use both 10G links at the same time too if your 6500 switches are in a VSS.  That would be nice :-)
Jody Lemoine

Alas, he's using WS-SUP720-3B supervisors... so no VSS. I agree, that would have been nice.
Craig Beck

Ah yes - didn't notice that! I read 3C somewhere but now I see that was your post.
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
Hello Sir,

Which fiber cables is better single mode or double mode.

Thanks
Craig Beck

Single-mode will allow much more bandwidth over longer distances.

For example, with single-mode OS1 cable you can achieve 10G up-to 40km, but only 550m over multi-mode OM4 cable.
Techrunner

ASKER
Sir,

Last thing I need to know whic LC connector I need for Switch 3750 10GB Speed.

Thank you experts,
Your help has saved me hundreds of hours of internet surfing.
fblack61
Jody Lemoine

Not sure what you mean by "LC connector" here. The SFP+ needed for the single-mode connection was outlined earlier. Are you looking for patch cable information?
Techrunner

ASKER
Gbic Module i.e  GLC-LH-SM and along with 10 GB C3KX-NM-10G
Jody Lemoine

Hey Samir. As we discussed earlier in this same topic, the 3750X needs a C3KX-NM-10G and one SFP-10G-LR for each fibre link connecting to it up to a maximum of two. The GLC-LH-SM is a GBIC and not appropriate for 10Gb connections at all.
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
Oopss...Sorry I didn't noticed that.

Thanks
Craig Beck

All SFPs use LC connectors.  The patch cord you need depends on what you're connecting the other end to.  If you're going direct from SFP to X2 module (for example) you'd need a LC-SC patch cord as the X2 module uses SC connectors.
Techrunner

ASKER
Hi,
single-mode G.652 fibre can be installed underground

Or is there specific cable specs for them.

Thanks
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
Craig Beck

Yes.  The type of cable isn't really important, but the way in which the cable is protected is.  You can buy fibre cable which is protected by steel-wire, for example.  This will go in a duct between two buildings.
Techrunner

ASKER
Hi,

How about Metal Free Armoured Fiber Cable.
Craig Beck

MFA is good... rats hate it!
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
:)
Techrunner

ASKER
How many cores fiber cable should between buildings abd floors
SOLUTION
Craig Beck

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Techrunner

ASKER
craigbeck and  jodylemoine, I would like thank you for extensive support.

Thanks
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
Techrunner

ASKER