10Giga Ethernet Link Between Two Buildings

Hello Experts,

We have two buildings 1KM away from each other.

At Building 1, we have 2 6500 Switches in existence.

At Building 2, we are going to install the switches such as follows

Ground Floor - 1 x WS-C3750X-24S-E
                        2 x WS C3560X- 48S-P
First Floor        2x WS-C3560X-48T-L

Second Floor   2xWS-C3560X-48T-L

All these switches will be connected to Cisco Fiber Switch ( Ground Floor)

Fiber Switch will be connected to Main Data Center Fiber 6500 Switch.
These switches will be in separate VTP domain.

I need the help
- How to 10 Giga Speed between Fiber Switch and 6500
- Selection of Fiber Optic Cable which is installed through underground
-  Fiber Gbic

Moreover, I dont vlans from 6500 to get copy on Building 2 Switch.


Thanks in advance
LVL 3
cciedreamerAsked:
Who is Participating?
 
Jody LemoineNetwork ArchitectCommented:
For a 1km link, you're looking at using single-mode G.652 fibre. If you were considering a 1GB connection, you could get away with multi-mode fibre with a 50-micron core, but you're not going to be able to do more than 400m with that at 10GB speeds.

The 3750X switch will need a C3KX-NM-10G module and a SFP-10G-LR SFP+ module. The Catalyst 6500 configuration will vary depending on what supervisors you're using. If you have supervisors that already support 10GB, then it's just a matter of getting an LR X2 or SFP+ module. If the supervisors don't support 10GB, then you'll either need to upgrade to supervisors that do or put in something like a WS-X6708-10G-3C module and a 10GBASE-LR X2 transceiver.
0
 
cciedreamerAuthor Commented:
Hello Sir,

Thanks for your response.

- How I can find out whether which supervisor is running on 6500
- WS-C3750X-24S-E has 24 fiber port, but I am not sure what is there speed.

Thanks
0
 
Jody LemoineNetwork ArchitectCommented:
You can do a "show module" on the Catalyst 6500 to get the details on all installed modules, including the supervisors.

The 3750X's on-board ports are going to be 1Gb, regardless of whether it's a copper or fibre-based switch. That's why the C3KX-NM-10G module is necessary. It plugs into the expansion slot on the right-hand side of the unit and adds 10Gb SFP+ capability to the switch. Once that's installed, you can put an SFP-10G-LR SFP+ module into it in order to connect to your fibre.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
cciedreamerAuthor Commented:
Just curious to know is will this be good lan design.

lan design
Result of Sh Module


sh module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL10425VXM
  2   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL10414EL3
  3   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     SAL1106GFFU
  5    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       SAL1109JJMR

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0019.2f57.1b74 to 0019.2f57.1b8b   2.5   12.2(14r)S5  12.2(18)SXD7 Ok
  2  0018.baaf.d9b8 to 0018.baaf.d9cf   2.5   12.2(14r)S5  12.2(18)SXD7 Ok
  3  001b.2a62.183c to 001b.2a62.186b  11.1   7.2(1)       8.3(0.156)RO Ok
  5  0016.46f9.2150 to 0016.46f9.2153   5.3   8.4(2)       12.2(18)SXD7 Ok

Mod Sub-Module                  Model              Serial        Hw     Status
--- --------------------------- ------------------ ------------ ------- -------
  1 Centralized Forwarding Card WS-F6700-CFC       SAL1107GYNF   2.1    Ok
  2 Centralized Forwarding Card WS-F6700-CFC       SAL1107GRJM   2.1    Ok
  5 Policy Feature Card 3       WS-F6K-PFC3B       SAL1102DRWS   2.3    Ok
  5 MSFC3 Daughterboard         WS-SUP720          SAL1109JM3P   2.6    Ok

Mod Online Diag Status
--- -------------------
  1 Pass
  2 Pass
  3 Pass
0
 
Jody LemoineNetwork ArchitectCommented:
The Supervisor Engine 720 3B doesn't have the 10Gb uplinks, unfortunately. You'll need to either upgrade to the VS-S720-10G-3C (or VS-S720-10G-3CXL) or add a 10Gb module (mentioned previously) to have 10Gb connectivity to the switch. Adding the module is the less expensive option, but if you're considering a supervisor upgrade anyway then moving to one that supports the uplinks you need makes sense.
0
 
cciedreamerAuthor Commented:
Sir,
Just to confirm is this the part number

SFP-10G-LR SFP+

How about LAN design ? Is it fine performance and redundancy wise
- Any specs for fiber cable to be installed under the ground ?
0
 
Jody LemoineNetwork ArchitectCommented:
The part number is just SFP-10G-LR. The cable specifications were listed in my first comment: single-mode G.652 fibre.

Not enough information has been provided to actually do a design analysis, but the basic connectivity looks reasonable. You might want to consider getting a second 3750X and stacking it with the first one to eliminate a single point of failure in your building two network.
0
 
cciedreamerAuthor Commented:
Sir,

Just a conclusion for the up links connectivity

Cisco 6500   - Get only SFP-10G-LR module
Cisco 3750  -  Get only SFP-10G-LR module
Fiber Cable  - Single Mode G.652

Right Sir ?
0
 
Jody LemoineNetwork ArchitectCommented:
Nope.

6500: Get WS-X6708-10G-3C and X2-10GB-LR
3750: Get C3KX-NM-10G and SFP-10G-LR
Fibre Cable: Single Mode G.652
0
 
Craig BeckCommented:
Just as a note (in addition to the excellent advice jodylemoine has already given), G.652 fibre may be referred to as OS1.  It's the same thing, just a different name by a different governing body.  It may be better though to refer to it as OS1 as that differentiates it from OS2 (which annoyingly is also referred to as G.652).

With the 6500 10G connectivity, you may need to use a X2 transceiver instead of a SFP, as jodylemoine quite rightly said - it just depends on which 10G module you purchase for your 6500.  However if you want to use SFPs on all of your switches for consistency you could use the X2-CVR transceiver module to adapt the X2 slot to accommodate a SFP.  Just be aware that doing this adds something else to go wrong.  It's a perfectly acceptable solution though.
0
 
cciedreamerAuthor Commented:
Thank you Sir

As per the design is concerned.
- We have numerous vlans on B1 Core Switches,
- For Building 2 I want to create separate vlans for them and default to B1 Cores
- I don't want B1 vlans to pass through or copy to B2 Switch.
0
 
Craig BeckCommented:
So use VTP transparent on all your switches, and prune all VLANs which don't need to be trunked to each switch stack.

For example, you want VLANs 101-200 in B1, VLANs 201-300 in B2 and VLANs 301-400 in B3.

At the core, create all VLANs.

On the trunk link to B1 you do:

interface PortChannel1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 101-200
 switchport trunk native vlan 50


but on the trunk to B2 you do:

interface PortChannel2
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 201-300
 switchport trunk native vlan 50


...and so on.

Make sense?
0
 
Jody LemoineNetwork ArchitectCommented:
If you want to be absolutely sure that this doesn't happen, you can set the 10Gb fibre interface on the 6500 as a routed interface. That way, no VLANs will pass at all. If you're going to do this, it makes more sense to configure the 3750X to be the default gateway for the building 2 VLANs and set the 10Gb fibre interfaces to be routed on both ends. Any traffic (if any) passing between the VLANs in building 2 can be handled locally by the 3750X and only traffic destined for the other building will be passed across the 10Gb fibre.
0
 
Jody LemoineNetwork ArchitectCommented:
On the 3750X, you can do:

interface TenGigabitEthernet1/0/0
 no switchport
 ip address 10.10.10.1 255.255.255.252

On the 6500, you would do:

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.2 255.255.255.252

(Interfaces and IPv4 addresses can be adjusted according to hardware configuration and addressing preference, of course.)

You can then define VLAN interfaces on the 3750X and put the default gateways for your VLANs there:

ip routing
!
interface VLAN50
 ip address 10.10.50.1 255.255.255.0
!
interface VLAN60
 ip address 10.10.60.1 255.255.255.0

The 3750X will be in control of your building 2 routing and will not leak VLANs to building 1, regardless of the VTP configuration on your switches in either building.

Then you can route between buildings using dynamic or static routing, and be absolutely sure that the VLANs in each of your buildings are separate.
0
 
cciedreamerAuthor Commented:
I read my mind. This is what I was thinking.

Can I also do in this way

B1 Core 1

interface vlan 50
ip address 10.10.50.1 255.255.255.0

interface Gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 101-200
 switchport trunk native vlan 50

but on the trunk to B2 you do:


interface vlan 50
ip address 10.10.50.2 255.255.255.0


interface gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 201-300
 switchport trunk native vlan 50
0
 
Jody LemoineNetwork ArchitectCommented:
This will work too, but it requires that you take steps to isolate your VTP configurations in each building, either by making all switches transparent or by setting different domains and passwords. It will also mean that all inter-VLAN routing between VLANs in building 2 will traverse your fibre to building 1.
0
 
cciedreamerAuthor Commented:
I want to keep inter vlan routing for B2 the local switch itself

Can I achieve this with HSRP becoz we have this

On the 6500,

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.2 255.255.255.252
0
 
Jody LemoineNetwork ArchitectCommented:
If you want to keep inter-VLAN routing for B2 on the local switch itself, then the configuration needs to be on the 3750X and not on the 6500, so there's no need for HSRP there.

If you want inter-VLAN routing for B2 to be on the remote switch, then you would go with craigbeck's suggestion of passive VLANs across the 10Gb fibre and put your HSRP (if needed) on the VLAN interfaces on the 6500.
0
 
cciedreamerAuthor Commented:
I am thinking to configure this and keep inter vlan routing on local switch.

However I have redundancy at B1, If core 1 goes down, how B2 Switch will route the traffic to Core2 without any manual intervention.
0
 
cciedreamerAuthor Commented:
I am thinking to configure this and keep inter vlan routing on local switch.

However I have redundancy at B1, If core 1 goes down, how B2 Switch will route the traffic to Core2 without any manual intervention.
0
 
Jody LemoineNetwork ArchitectCommented:
If you have two 10Gb fibres running, one to each core, you can just run a subnet on each link and use a routing protocol to manage the connectivity:

Catalyst 3750X:

interface TenGigabitEthernet1/0/0
 no switchport
 ip address 10.10.10.2 255.255.255.252
!
interface TenGigabitEthernet1/0/1
 no switchport
 ip address 10.10.10.6 255.255.255.252
!
router eigrp 1010
 no auto-summary
 network 10.10.10.0 0.0.0.7
 eigrp stub connected

Catalyst 6500

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.1 255.255.255.252
!
interface TenGigabitEthernet1/1
 no switchport
 ip address 10.10.10.5 255.255.255.252
!
router eigrp 1010
 no auto-summary
 network 10.10.10.0 0.0.0.7

Should either of the fibres go down, the routing protocol will route traffic over the other.
0
 
cciedreamerAuthor Commented:
I guess I should I  go with this Thanks

Just one 1 more query, I am going to place 15 U Wall Mounted Cabinet on each floor

Ground Floor Will hold 3 Switch

First floor will hold 2 x 48
Second floor will hold 2 x 48

Room Space Width 1m x Depth 1.5 m

You would like to shed some light, if thats fine.
0
 
Jody LemoineNetwork ArchitectCommented:
None of those switches are more than a half metre from front to back, so that should be more than adequate.
0
 
cciedreamerAuthor Commented:
Thank You Sir,

I will just make final plan if there is anything I'll update you.
0
 
Craig BeckCommented:
Agree with jodylemoine - L3 distribution layer is the way to go.
0
 
cciedreamerAuthor Commented:
Thanks craigbeck
0
 
Craig BeckCommented:
You may be able to use both 10G links at the same time too if your 6500 switches are in a VSS.  That would be nice :-)
0
 
Jody LemoineNetwork ArchitectCommented:
Alas, he's using WS-SUP720-3B supervisors... so no VSS. I agree, that would have been nice.
0
 
Craig BeckCommented:
Ah yes - didn't notice that! I read 3C somewhere but now I see that was your post.
0
 
cciedreamerAuthor Commented:
Hello Sir,

Which fiber cables is better single mode or double mode.

Thanks
0
 
Craig BeckCommented:
Single-mode will allow much more bandwidth over longer distances.

For example, with single-mode OS1 cable you can achieve 10G up-to 40km, but only 550m over multi-mode OM4 cable.
0
 
cciedreamerAuthor Commented:
Sir,

Last thing I need to know whic LC connector I need for Switch 3750 10GB Speed.

Thank you experts,
0
 
Jody LemoineNetwork ArchitectCommented:
Not sure what you mean by "LC connector" here. The SFP+ needed for the single-mode connection was outlined earlier. Are you looking for patch cable information?
0
 
cciedreamerAuthor Commented:
Gbic Module i.e  GLC-LH-SM and along with 10 GB C3KX-NM-10G
0
 
Jody LemoineNetwork ArchitectCommented:
Hey Samir. As we discussed earlier in this same topic, the 3750X needs a C3KX-NM-10G and one SFP-10G-LR for each fibre link connecting to it up to a maximum of two. The GLC-LH-SM is a GBIC and not appropriate for 10Gb connections at all.
0
 
cciedreamerAuthor Commented:
Oopss...Sorry I didn't noticed that.

Thanks
0
 
Craig BeckCommented:
All SFPs use LC connectors.  The patch cord you need depends on what you're connecting the other end to.  If you're going direct from SFP to X2 module (for example) you'd need a LC-SC patch cord as the X2 module uses SC connectors.
0
 
cciedreamerAuthor Commented:
Hi,
single-mode G.652 fibre can be installed underground

Or is there specific cable specs for them.

Thanks
0
 
Craig BeckCommented:
Yes.  The type of cable isn't really important, but the way in which the cable is protected is.  You can buy fibre cable which is protected by steel-wire, for example.  This will go in a duct between two buildings.
0
 
cciedreamerAuthor Commented:
Hi,

How about Metal Free Armoured Fiber Cable.
0
 
Craig BeckCommented:
MFA is good... rats hate it!
0
 
cciedreamerAuthor Commented:
:)
0
 
cciedreamerAuthor Commented:
How many cores fiber cable should between buildings abd floors
0
 
Craig BeckCommented:
That depends on how many links you want to connect between buildings.  In your scenario a 4-core would do what you need.

I usually specify at least:

Total Cores = Required Cores + (Required Cores / 2)

So, if I need 2 'links' I'd usually require 4 cores (as each link needs 2 cores unless you use BX transceivers).  Using the formula that's 6 cores.  This will give you spare cores if you need them.

Obviously though if it's not going to cost much more you could specify say a 12-core fibre if you only need a 6-core.  It's always better to have more than you need.
0
 
cciedreamerAuthor Commented:
craigbeck and  jodylemoine, I would like thank you for extensive support.

Thanks
0
 
cciedreamerAuthor Commented:
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.