We help IT Professionals succeed at work.

10Giga Ethernet Link Between Two Buildings

929 Views
Last Modified: 2013-12-26
Hello Experts,

We have two buildings 1KM away from each other.

At Building 1, we have 2 6500 Switches in existence.

At Building 2, we are going to install the switches such as follows

Ground Floor - 1 x WS-C3750X-24S-E
                        2 x WS C3560X- 48S-P
First Floor        2x WS-C3560X-48T-L

Second Floor   2xWS-C3560X-48T-L

All these switches will be connected to Cisco Fiber Switch ( Ground Floor)

Fiber Switch will be connected to Main Data Center Fiber 6500 Switch.
These switches will be in separate VTP domain.

I need the help
- How to 10 Giga Speed between Fiber Switch and 6500
- Selection of Fiber Optic Cable which is installed through underground
-  Fiber Gbic

Moreover, I dont vlans from 6500 to get copy on Building 2 Switch.


Thanks in advance
Comment
Watch Question

Network Architect
CERTIFIED EXPERT
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
Hello Sir,

Thanks for your response.

- How I can find out whether which supervisor is running on 6500
- WS-C3750X-24S-E has 24 fiber port, but I am not sure what is there speed.

Thanks
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
You can do a "show module" on the Catalyst 6500 to get the details on all installed modules, including the supervisors.

The 3750X's on-board ports are going to be 1Gb, regardless of whether it's a copper or fibre-based switch. That's why the C3KX-NM-10G module is necessary. It plugs into the expansion slot on the right-hand side of the unit and adds 10Gb SFP+ capability to the switch. Once that's installed, you can put an SFP-10G-LR SFP+ module into it in order to connect to your fibre.

Author

Commented:
Just curious to know is will this be good lan design.

lan design
Result of Sh Module


sh module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL10425VXM
  2   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL10414EL3
  3   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     SAL1106GFFU
  5    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       SAL1109JJMR

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0019.2f57.1b74 to 0019.2f57.1b8b   2.5   12.2(14r)S5  12.2(18)SXD7 Ok
  2  0018.baaf.d9b8 to 0018.baaf.d9cf   2.5   12.2(14r)S5  12.2(18)SXD7 Ok
  3  001b.2a62.183c to 001b.2a62.186b  11.1   7.2(1)       8.3(0.156)RO Ok
  5  0016.46f9.2150 to 0016.46f9.2153   5.3   8.4(2)       12.2(18)SXD7 Ok

Mod Sub-Module                  Model              Serial        Hw     Status
--- --------------------------- ------------------ ------------ ------- -------
  1 Centralized Forwarding Card WS-F6700-CFC       SAL1107GYNF   2.1    Ok
  2 Centralized Forwarding Card WS-F6700-CFC       SAL1107GRJM   2.1    Ok
  5 Policy Feature Card 3       WS-F6K-PFC3B       SAL1102DRWS   2.3    Ok
  5 MSFC3 Daughterboard         WS-SUP720          SAL1109JM3P   2.6    Ok

Mod Online Diag Status
--- -------------------
  1 Pass
  2 Pass
  3 Pass
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
The Supervisor Engine 720 3B doesn't have the 10Gb uplinks, unfortunately. You'll need to either upgrade to the VS-S720-10G-3C (or VS-S720-10G-3CXL) or add a 10Gb module (mentioned previously) to have 10Gb connectivity to the switch. Adding the module is the less expensive option, but if you're considering a supervisor upgrade anyway then moving to one that supports the uplinks you need makes sense.

Author

Commented:
Sir,
Just to confirm is this the part number

SFP-10G-LR SFP+

How about LAN design ? Is it fine performance and redundancy wise
- Any specs for fiber cable to be installed under the ground ?
Jody LemoineNetwork Architect
CERTIFIED EXPERT
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
Sir,

Just a conclusion for the up links connectivity

Cisco 6500   - Get only SFP-10G-LR module
Cisco 3750  -  Get only SFP-10G-LR module
Fiber Cable  - Single Mode G.652

Right Sir ?
Jody LemoineNetwork Architect
CERTIFIED EXPERT
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
Thank you Sir

As per the design is concerned.
- We have numerous vlans on B1 Core Switches,
- For Building 2 I want to create separate vlans for them and default to B1 Cores
- I don't want B1 vlans to pass through or copy to B2 Switch.
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
So use VTP transparent on all your switches, and prune all VLANs which don't need to be trunked to each switch stack.

For example, you want VLANs 101-200 in B1, VLANs 201-300 in B2 and VLANs 301-400 in B3.

At the core, create all VLANs.

On the trunk link to B1 you do:

interface PortChannel1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 101-200
 switchport trunk native vlan 50


but on the trunk to B2 you do:

interface PortChannel2
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 201-300
 switchport trunk native vlan 50


...and so on.

Make sense?
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
If you want to be absolutely sure that this doesn't happen, you can set the 10Gb fibre interface on the 6500 as a routed interface. That way, no VLANs will pass at all. If you're going to do this, it makes more sense to configure the 3750X to be the default gateway for the building 2 VLANs and set the 10Gb fibre interfaces to be routed on both ends. Any traffic (if any) passing between the VLANs in building 2 can be handled locally by the 3750X and only traffic destined for the other building will be passed across the 10Gb fibre.
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
On the 3750X, you can do:

interface TenGigabitEthernet1/0/0
 no switchport
 ip address 10.10.10.1 255.255.255.252

On the 6500, you would do:

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.2 255.255.255.252

(Interfaces and IPv4 addresses can be adjusted according to hardware configuration and addressing preference, of course.)

You can then define VLAN interfaces on the 3750X and put the default gateways for your VLANs there:

ip routing
!
interface VLAN50
 ip address 10.10.50.1 255.255.255.0
!
interface VLAN60
 ip address 10.10.60.1 255.255.255.0

The 3750X will be in control of your building 2 routing and will not leak VLANs to building 1, regardless of the VTP configuration on your switches in either building.

Then you can route between buildings using dynamic or static routing, and be absolutely sure that the VLANs in each of your buildings are separate.

Author

Commented:
I read my mind. This is what I was thinking.

Can I also do in this way

B1 Core 1

interface vlan 50
ip address 10.10.50.1 255.255.255.0

interface Gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 101-200
 switchport trunk native vlan 50

but on the trunk to B2 you do:


interface vlan 50
ip address 10.10.50.2 255.255.255.0


interface gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 201-300
 switchport trunk native vlan 50
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
This will work too, but it requires that you take steps to isolate your VTP configurations in each building, either by making all switches transparent or by setting different domains and passwords. It will also mean that all inter-VLAN routing between VLANs in building 2 will traverse your fibre to building 1.

Author

Commented:
I want to keep inter vlan routing for B2 the local switch itself

Can I achieve this with HSRP becoz we have this

On the 6500,

interface TenGigabitEthernet1/0
 no switchport
 ip address 10.10.10.2 255.255.255.252
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
If you want to keep inter-VLAN routing for B2 on the local switch itself, then the configuration needs to be on the 3750X and not on the 6500, so there's no need for HSRP there.

If you want inter-VLAN routing for B2 to be on the remote switch, then you would go with craigbeck's suggestion of passive VLANs across the 10Gb fibre and put your HSRP (if needed) on the VLAN interfaces on the 6500.

Author

Commented:
I am thinking to configure this and keep inter vlan routing on local switch.

However I have redundancy at B1, If core 1 goes down, how B2 Switch will route the traffic to Core2 without any manual intervention.

Author

Commented:
I am thinking to configure this and keep inter vlan routing on local switch.

However I have redundancy at B1, If core 1 goes down, how B2 Switch will route the traffic to Core2 without any manual intervention.
Jody LemoineNetwork Architect
CERTIFIED EXPERT
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
I guess I should I  go with this Thanks

Just one 1 more query, I am going to place 15 U Wall Mounted Cabinet on each floor

Ground Floor Will hold 3 Switch

First floor will hold 2 x 48
Second floor will hold 2 x 48

Room Space Width 1m x Depth 1.5 m

You would like to shed some light, if thats fine.
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
None of those switches are more than a half metre from front to back, so that should be more than adequate.

Author

Commented:
Thank You Sir,

I will just make final plan if there is anything I'll update you.
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
Agree with jodylemoine - L3 distribution layer is the way to go.

Author

Commented:
Thanks craigbeck
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
You may be able to use both 10G links at the same time too if your 6500 switches are in a VSS.  That would be nice :-)
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
Alas, he's using WS-SUP720-3B supervisors... so no VSS. I agree, that would have been nice.
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
Ah yes - didn't notice that! I read 3C somewhere but now I see that was your post.

Author

Commented:
Hello Sir,

Which fiber cables is better single mode or double mode.

Thanks
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
Single-mode will allow much more bandwidth over longer distances.

For example, with single-mode OS1 cable you can achieve 10G up-to 40km, but only 550m over multi-mode OM4 cable.

Author

Commented:
Sir,

Last thing I need to know whic LC connector I need for Switch 3750 10GB Speed.

Thank you experts,
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
Not sure what you mean by "LC connector" here. The SFP+ needed for the single-mode connection was outlined earlier. Are you looking for patch cable information?

Author

Commented:
Gbic Module i.e  GLC-LH-SM and along with 10 GB C3KX-NM-10G
Jody LemoineNetwork Architect
CERTIFIED EXPERT

Commented:
Hey Samir. As we discussed earlier in this same topic, the 3750X needs a C3KX-NM-10G and one SFP-10G-LR for each fibre link connecting to it up to a maximum of two. The GLC-LH-SM is a GBIC and not appropriate for 10Gb connections at all.

Author

Commented:
Oopss...Sorry I didn't noticed that.

Thanks
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
All SFPs use LC connectors.  The patch cord you need depends on what you're connecting the other end to.  If you're going direct from SFP to X2 module (for example) you'd need a LC-SC patch cord as the X2 module uses SC connectors.

Author

Commented:
Hi,
single-mode G.652 fibre can be installed underground

Or is there specific cable specs for them.

Thanks
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
Yes.  The type of cable isn't really important, but the way in which the cable is protected is.  You can buy fibre cable which is protected by steel-wire, for example.  This will go in a duct between two buildings.

Author

Commented:
Hi,

How about Metal Free Armoured Fiber Cable.
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014

Commented:
MFA is good... rats hate it!

Author

Commented:
:)

Author

Commented:
How many cores fiber cable should between buildings abd floors
some oneNetwork Architect
CERTIFIED EXPERT
Top Expert 2014
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
craigbeck and  jodylemoine, I would like thank you for extensive support.

Thanks
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.