Redundant Switch Setup - Questions
I am designing a network for a new building. There will be a main server room and 5 IDFs. In each IDF I want to have as much redundancy as possible, so in each IDF there will be 2 XG/aggregate switches. The idea is that no matter how many 48-port switches are in a cabinet, EACH one will connect to 2 XG switches (one fiber cable to each). Then, to get from one IDF to the next, each XG switch in turn will have a dedicated fiber connection to the other XG switch in the next cabinet. The IDFs are connected in a star topology, meaning one "central" IDF has 2 fiber connections to each of the other 4 IDFs - one in each corner of the building. So the redundancy here is that every 48 port switch itself is connected to 2 XG switches, and in turn every IDF has 2 paths to the central IDF.
Here is a partial diagram to illustrate what I mean:
The red lines indicate fiber connections.
The blue lines also indicate fiber connections but may be overkill (Within each IDF, a single fiber cable connects the 2 XG switches to each other)
I am using RSTP on all switches, with the default settings except I made the following change: I gave each of the 48-port switches a priority of "32,768" while all of the 16-port switches have a priority of "16,384". My assumption is that STP will choose each XG switch over a 48 port switch when building out its path (but I suspect there is more to it than that)
A few questions:
1. Is "Priority" the same as "Cost"? meaning, the faster switches should have a LOWER number (as I indicated above). Is this correct?
2. I started out with one fiber connection between each IDF. At some point I plugged in all of the second (redundant) connections between IDFs and then I noticed some of my servers went offline. Maybe that's because STP was busy re-calculating the new path(s) and maybe I didn't wait long enough (I waited about 20 minutes). As soon as I unplugged one of the redundant paths the servers come back online. I suspect this is because when it comes to Ethernet/IP networking you should not have more than one path to anything. So if that's the case, how can I program/design a redundant network? (In other words, what additional programming must be done to tell the switches to use one path or the other, don't try to use both? I thought the whole point of STP was to calculate and elect a single path and stick with it until either (a) something goes offline or (b) there is a topology change)
3. Not shown in the diagram is the actual main server room. Should the switches in the server room have an even lower cost than all others? (assuming most traffic is between a client and a server (or router))
4. I know that if I had 2 cables connecting Switch A to Switch B, I could select both ports and create a LAG. The LAG will provide a bigger pipe (not necessary in this scenario), load balancing and also redundancy (and "redundancy" is what I am after here). But in my scenario, I don't have 2 fiber connections going to a single XG switch. Instead, I have one fiber going to XG Switch number 1 and another fiber going to XG Switch number 2. Can the 2 XG switches in a sense form the LAG? (Or is a LAG only allowed between 2 devices).
My concern is that in an attempt to add redundancy to my network, I have created too many paths and loops.
Here is a partial diagram to illustrate what I mean:
The red lines indicate fiber connections.
The blue lines also indicate fiber connections but may be overkill (Within each IDF, a single fiber cable connects the 2 XG switches to each other)
I am using RSTP on all switches, with the default settings except I made the following change: I gave each of the 48-port switches a priority of "32,768" while all of the 16-port switches have a priority of "16,384". My assumption is that STP will choose each XG switch over a 48 port switch when building out its path (but I suspect there is more to it than that)
A few questions:
1. Is "Priority" the same as "Cost"? meaning, the faster switches should have a LOWER number (as I indicated above). Is this correct?
2. I started out with one fiber connection between each IDF. At some point I plugged in all of the second (redundant) connections between IDFs and then I noticed some of my servers went offline. Maybe that's because STP was busy re-calculating the new path(s) and maybe I didn't wait long enough (I waited about 20 minutes). As soon as I unplugged one of the redundant paths the servers come back online. I suspect this is because when it comes to Ethernet/IP networking you should not have more than one path to anything. So if that's the case, how can I program/design a redundant network? (In other words, what additional programming must be done to tell the switches to use one path or the other, don't try to use both? I thought the whole point of STP was to calculate and elect a single path and stick with it until either (a) something goes offline or (b) there is a topology change)
3. Not shown in the diagram is the actual main server room. Should the switches in the server room have an even lower cost than all others? (assuming most traffic is between a client and a server (or router))
4. I know that if I had 2 cables connecting Switch A to Switch B, I could select both ports and create a LAG. The LAG will provide a bigger pipe (not necessary in this scenario), load balancing and also redundancy (and "redundancy" is what I am after here). But in my scenario, I don't have 2 fiber connections going to a single XG switch. Instead, I have one fiber going to XG Switch number 1 and another fiber going to XG Switch number 2. Can the 2 XG switches in a sense form the LAG? (Or is a LAG only allowed between 2 devices).
My concern is that in an attempt to add redundancy to my network, I have created too many paths and loops.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
noci and atlas, big thanks for the crash course and recommendations. This past weekend I was able to deploy all of my switches. I followed your advice to set STP priorities and the network is running better than ever. I am not able to make the server room the hub (although that makes sense) simply because of the physical location. (I'd have run over 1,000 feet of fiber cable, never mind the challenge of running it all overhead above drop ceilings and warehouse racks). Thanks again!
ASKER
Thanks for your reply. Very helpful. Yes, I am sure STP is enabled on all switches. I'll be completing this project over the next week. Once all of the new switches are in position I will connect all segments together again and reply back with my results. Regarding the LAG - looks like the EdgeSwitches do not support stacking, so I'll set the second (redundant) XG16 in each segment to a higher cost than the first one.
atlas,
Also big thanks for your reply. Here is a more complete view of the network diagram. I referred to it as a star topology because one IDF (the yellow one in this diagram) is the central hub. The server room (shaded in blue) is where the router and firewall live. So these two segments (server room and central IDF) are the primary path for everything on the network. Notice the outer segments do not connect to each other. But each has a dual/redundant path to the central IDF, which in turn has a dual/redundant path to the server room.
Given this scenario, would it be best to give the server room switches the lowest cost, and then the central IDF the second-lowest cost?