Solved

move 1 gig fiber connectiont o trunked 2 gig?

Posted on 2011-09-25
17
549 Views
Last Modified: 2012-05-12
we're using brocade/foundry FWS series switches. it's a flat network. we have a gbic in the first port of the 4 port gbic module on each of the switches.

I would like to add a 2nd in as a 2 gig trunk. we have the gbics in and the fiber connected.

here's the tough part: I can only have myself touching the switches, no one else is available to help. and they are 20 minutes apart connected only by this single link. so if it goes down, i have to drive 25 minutes to the other site. if I have to do this once, that's fine. But to do it twice is not good, let alone more.

question 1: is this possible without downtime? they are very far apart and I can't easily run to the other one, nor can I have someone at both sites at once.

question 2: if there has to be temporary downtime while I drop the existing trunk, is this what has to happen:

1. drop trunk setting on one site (do remotely from site I'm at).
2. drop trunk on other site (where I'm at)
3. recreate trunk at site I'm at.
4. drive to remote site and recreate trunk there as well.

Or can I do this:

1. at site I'm at, drop trunk.
2. recreate 2 port trunk. link is now back up (even though only using 1 port at remote site)
3. drive to remote site, drop trunk
4. recreate 2 port trunk at remote site.

keep in mind, these switches are connected via a private L2 vlan, but not physically connected directly. so it seems to me that I can drop it at one site and recreate and it will go right back up, but still at 1 gig. does that make sense?

lastly, whatever the best way, please provide commands to a newbie. gui or CLI is fine.

Thanks!
0
Comment
Question by:MrVault
  • 8
  • 8
17 Comments
 
LVL 3

Expert Comment

by:weedhell
Comment Utility
do you have an extra laptop with internet connection in some of the 2 places? you could use some remote software so that you could be cloning yourself ;) teamviewer should be a good choice...
0
 

Author Comment

by:MrVault
Comment Utility
Though about that nd except the connection to it  via team viewer would go over this link
0
 
LVL 8

Expert Comment

by:gsmartin
Comment Utility
When you break a trunk, the connectivity only goes down briefly for about 7 or so pings (5 to 10 seconds).  Connectivity will come backup eventhough the trunk is down on one side.  So in other words, the connection between switches will still function using standard Layer 2/3 protocols.

Also, when adding or removing the trunk you need to use 'Trunk Deploy' (in config mode) for the changes to take affect.  So you will be able to configure everything remotely and then drive to the site to add the additional circuit.

Also, you should make sure port settings (speed/duplex) and VLANs tags match on both circuits used for the LACP trunk.  

Config Mode Commands:
 trunk ethe 0/1/47 to 0/1/48
 trunk deploy  (The config will show, but won't be effective until you run this command or the switch is rebooted)

Enable mode Commands:
 Sh Trunk
0
 
LVL 8

Expert Comment

by:gsmartin
Comment Utility
Fyi... Outage is actually about 20 to 30 seconds.
0
 
LVL 8

Assisted Solution

by:gsmartin
gsmartin earned 500 total points
Comment Utility
Example of Some Tests:
telnet@LB_FWS624G-POE_IDF2a(config)#no trunk ethe 0/1/23 to 0/1/24
Trunk will be removed in next trunk deploy.
telnet@LB_FWS624G-POE_IDF2a(config)#trunk deploy

OUTAGE:
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128

telnet@LB_FWS624G-POE_IDF2a(config)#sh trunk
Configured trunks:

None
Operational trunks:

None
telnet@LB_FWS624G-POE_IDF2a(config)#trunk ethe 0/1/23 to 0/1/24
Trunk will be created in next trunk deploy.


telnet@LB_FWS624G-POE_IDF2a(config)#trunk deploy

OUTAGE:
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128
Reply from 10.1.254.20: bytes=32 time=6ms TTL=128
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 10.1.254.20: bytes=32 time=4ms TTL=128
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128
Reply from 10.1.254.20: bytes=32 time=3ms TTL=128
Reply from 10.1.254.20: bytes=32 time=1ms TTL=128

telnet@LB_FWS624G-POE_IDF2a(config)#sh trunk
Configured trunks:

Trunk ID: 23
Hw Trunk ID: 1
Ports_Configured: 2
Primary Port Monitored: Jointly

Ports         0/1/23  0/1/24
Port Names    none    none
Port_Status   enable  enable
Monitor       off     off
Rx Mirr Port  N/A     N/A
Tx Mirr Port  N/A     N/A
Monitor Dir   N/A     N/A

Operational trunks:

Trunk ID: 23
Hw Trunk ID: 1
Duplex: Full
Speed: 1G
Tag: Yes
Priority: level0
Active Ports: 1

Ports         0/1/23     0/1/24
Link_Status   active     down
port_state    Forward    Blocked
0
 

Author Comment

by:MrVault
Comment Utility
Here's the trouble.

I just connected switch B, port 2 to the fiber switch and created a trunk between ports 1 and 2 and ran trunk deploy. however, the 2nd port is not lighting up. I can still ping the other side. but as your printout shows, I have port/link states of active and down. If I switch the fiber connection sides (switch A and B of the cable), they both go to UP, but I can no longer ping.

I have not connected both ports on the switch at the other side, but both ports over there are already in a trunk together.

what gives? Also, how do I verify the trunk is in the right vlan?
0
 

Author Comment

by:MrVault
Comment Utility
I read online that all ports in trunks on both sides have to be disconnected. so essentially it's saying:

disconnect all ports, dropping existing link.
create trunk on side 1 and run trunk deploy. somehow verify trunk is in right vlan.
create trunk on side 2 and run trunk deploy. verify vlan.
connect cables on side 2.
connect cables on side 1.

here's the problem. side 1 and side 2 are a 25 minute drive away! so in total we're down for minimum 75 minutes, but more likely 120 minutes. that sucks.

I don't see why it doesn't work to create the trunk on one side and only 1 link will be up at a time. then go to other side, create trunk and connect and now both links are up.
0
 
LVL 8

Expert Comment

by:gsmartin
Comment Utility
You should be able to connect the second circuit, but disable the interface.  That way LACP won't try to establish.  LACP can only work with at least two uplinks in order to communicate.  So the key is keep the secondary circuit down until you have gone to the other site to connect the other cable.  At that point, you can re-enable the interface on the opposite side, which will then bring up the LACP trunk.  Then enable the new circuit's interface and the trunk come online.    

Just to reiterate, disable the new (secondary) network interfaces on both switches; until you are ready to enable the LACP trunk.  Enabling the secondary interface will force LACP vs standard Layer 2 connectivity.

Global VLAN Commands:
sh vlan
sh vlan 1
sh vlan e 0/1/47

Make sure that all interfaces are tagging the same VLANs.  Use the 'sh vlan eth 0/1/47' command to see VLANs the interface is tied to.  You can also see this from the running config 'sh run'.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Accepted Solution

by:
MrVault earned 0 total points
Comment Utility
I'm a bit confused. Are you saying to shutdown the port before creating the trunk? And to do this on both switches? So is it this:

disable port 2 on both switches
plug in cable to port 2 on both switches
create trunks on both switches with ports 1 and 2 on both switches
enable port 2 on remote switch. enable port 2 on switch onsite.

and voila I should have a 2 port trunk between the switches?

If so, I have a couple follow up questions:

If I do not create the trunk on 1 switch and do not plug the 2nd port in on that switch, can I create the trunk on the 1st switch and have the cable plugged in? that is what I have done thus far:

switch 1 has trunk created with 2 ports and 1 of those ports has fiber connection.
switch 2 has a single port connected via fiber
connection between the two switches is great.

I added gbic to 2nd fiber port on switch 2
created 2 port trunk on switch 2
connection stays up
plug in fiber cable into switch 2 (but not into switch 1)
gbic port lights up solid green, but can't ping across ports.
if I reverse the fiber cable pieces (A and B wire), the port stops lighting up and ping works again.

I should mention that both switches connect to our network provider's fabric, not directly into each other. do I possible have to ask them to make a trunk? they said it's a private L2 vlan our links are in. they have a 10gig connection between their two fabric switches.


0
 
LVL 8

Expert Comment

by:gsmartin
Comment Utility
On Switch 2, disable port 2 until you have already reconnected port 2 on switch 1.  Testing the connectivity between you and your provider is fine, but once you have verified your connection is hot; disable it.  
Sorry, I just realized I’ve overlooked a very important component.  You need to check with your service provider(s) to see if static LAG is supported?  You may need to request to have it supported.  Also, do you have separate service providers for each circuit?  LAG support will most likely depend on their underlying network protocol s and equipment being used; I assume the private Layer 2 VLAN is Ethernet or potentially VPLS?  
Typically, in this scenario the Providers End switch may not by default be completely transparent, which is the reason why, in some cases, LAG won’t work initially by default through a carrier.  Usually it’s available upon request and requires provisioning (configuration).   This could become more complicated depending on the following:  Tier of the provider, protocols used and supported (LAG/LACP?), Will this connection go through multiple switches/networks to get to your other switch (how many switches or networks between points?), etc… ?   As you can see there could be some challenges.  
Anyway, you will need to work with your provider(s) to get the details worked out on ensuring LAG can be supported (all the way through each end).  Now, assuming everything gets provisioned correctly (unless it’s already completed) you could proceed with the connectivity plan.  
0
 

Author Comment

by:MrVault
Comment Utility
Hi. Well, the world collapsed on me yesterday. I tried your instructions and Brocades and both times the link came up fine but after a minute or two starting going up and down. After trying to fix it I just undid the settings. My monitoring software alerted me connections to the other side were not available and I couldn't have that. Turns out though, a loop was created and every server that had an iSCSI volume dropped the volume! All came back up after a reboot except one and after getting 1 hour of sleep last night and letting diskpart scan it for 8.5 hours the volume finally came back.

Point being, I think I'm out of my league here and need to bring in professional services. Below is what our provider who gives us the back-link between cities said. Any idea on what they're saying?

these 1G links are not bonded and can not be on our end.

I can see you have apparently tried to bond or otherwise run both links concurrently
as some of our gear has been indicating a flapping of arp addresses between interfaces.

If you absolutely must bond these, the only advise we can give if your equipment can
support it would be to run two tunnels between your switches, and bond those interfaces.

There is no guarantee if it works how much throughput you'll really get, but it is the
only solution I could think of that might work for you provided your gear supports it.

What you really need here is a higher capacity connection.

0
 
LVL 8

Assisted Solution

by:gsmartin
gsmartin earned 500 total points
Comment Utility
Sorry for the delayed response.  I've out of town on a project and just got back.  

Sorry to hear about your situation.  You don't need a higher capacity circuit, eventhough by bonded the circuits it will provide you more capacity.  One the main points is the High Availability between the circuits in case one goes down.  

The main question is how Transparent are these circuits?  This will depend on the provider and the type of service they are providing.  E.g. An EPL (Ethernet Private Line) has more Transparency than an EVPL (Ethernet Virtual Private Line).  The EVPL is required to either peer or drop most of the Layer 2 Control Protocols.

I will have to see what they are referring to in regards to the two tunnels that could allow you to bond.  Bonding would most likely takeplace using LAG (LACP), but not sure what protocol he is reffering to for the tunnels, maybe VPLS; not sure since I haven't used it myself.

FYI... "Virtual Private LAN Service (VPLS) is a way to provide Ethernet based multipoint to multipoint communication over IP/MPLS networks. It allows geographically dispersed sites to share an Ethernet broadcast domain by connecting sites through pseudo-wires. The technologies that can be used as pseudo-wire can be Ethernet over MPLS, L2TPv3 or even GRE.

VPLS is a virtual private network (VPN) technology. In contrast to L2TPv3, which allows only point-to-point layer 2 tunnels, VPLS allows any-to-any (multipoint) connectivity.

In a VPLS, the local area network (LAN) at each site is extended to the edge of the provider network. The provider network then emulates a switch or bridge to connect all of the customer LANs to create a single bridged LAN."

VPLS is more what I would expect to be used by your provider.  If you can more specifics from them on what protocols they use  on what layer and what tunneling protocols do they recommend to meet your objective.


0
 

Author Comment

by:MrVault
Comment Utility
Thanks. I'm going to schedule a call between our ISP and our network professional services contractor. Unfortunately I'm way out of my league now. I will definitely update when I hear back. They said something to the effect of "we thought you just wanted a redundant link. they are on different frequencies and switches altogether. you need to bond your circuits....yada yada yada"

I know alot about technology, but networking is probably my biggest flaw. Once you get beyond the basics, I quickly fall behind.
0
 
LVL 8

Expert Comment

by:gsmartin
Comment Utility
No worries. Given how much Information technology is out there no one can know everything, especially as you get into the more complicated areas of networking and Telco circuits.

FYI... The issue you experienced the other day in LAN scenarios, when not creating a LAG, requires Spanning Tree to prevent exactly what you encountered.  Spanning Tree will put one of the two link's interface in block mode, which prevents the behavior that you experienced.   If the primary circuit fails Spanning Tree will then re-enable the secondary circuit and put in listening mode.    

Anyway, LAG is more common these days, because you get better use out of your uplink circuits.

It's good that you set up a call with your contractor and provider to verify wither you will be able to use LAG.  Also, for redundancy purposes you should make sure that your circuits are terminating into separate switches within your provider’s network on both ends.  This will ensure that both circuits are not affected at the same time when then are doing maintenance or experience an outage.

Out of curiosity, who is your service provider for the two circuits?
0
 

Author Closing Comment

by:MrVault
Comment Utility
our ISP messed up this process, but I think the directions are accurate.
0
 

Author Comment

by:MrVault
Comment Utility
Turns out they didn't bond the ports and vlan traffic from us across the various hops between the datacenters. That's why it failed.
0
 
LVL 8

Expert Comment

by:gsmartin
Comment Utility
Gotta love Service providers!   Glad to here the issue is resolved.
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Join & Write a Comment

I recently attended Cisco Live! in Las Vegas, a conference that boasted over 28,000 techies in attendance, and a week of hands-on learning hosted by a solid partner with which Concerto goes to market.  Every year, Cisco displays cutting-edge technol…
PRTG Network Monitor lets you monitor your bandwidth usage, so you know who is using up your bandwidth, and what they're using it for.
Viewers will learn how to connect to a wireless network using the network security key. They will also learn how to access the IP address and DNS server for connections that must be done manually. After setting up a router, find the network security…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now