A new customer building consists of 7-8 floors, 14-16 IDFs and an MDF split between two sides of the same floor. All equipment has already been procured, which consists of Cisco Catalyst 9K (fully Layer-3 licensed) throughout.
We had a design call with them last week, and their intended logical design was to make everything Layer 2 in this new building, pulling all routing and security back over the fiber connections across the street to Nexus 7K switches currently serving as the core for everything. (So, all VLANs/subnets would extend outside of the building back to the core across the street.)
The logic here would be that they want to have a central policy point/device through which all traffic would traverse and thus be inspected. They're aware of Cisco ISE/SGT technology, but seem at this point convinced that the best approach would be NOT to Layer 3 segment the new building, even at the core pair in the MDF.
My initial input on the call was to heavily advise Layer 3 at the building "core" (two Cisco Catalyst 9600 chassis in a logical pair), if not also to roll out a routed access design (L3 at the edge).
The guy that recommended this "all L2 in the new building" design comes from a security background, and I've seen this approach quite a bit, trying to hairpin all traffic back to a pair of firewalls. This doesn't make much sense to me from a design and scale standpoint. Maybe it'd work in smaller data centers, but for new campus building location?
I'm looking for input to either combat or support this customer design.
- If we implemented a routed design (within the building and back to the core across the street), what would we leverage for centralized policy. Would this be a Cisco ISE/SGT design?
- Do we agree that extending potentially 100 subnets/VLANs (thus, broadcast domains) outside of a 600-uesr building back to a different site would be a ridiculous risk and failure domain?
- Anything else?
I do not agree that pulling 600 users and 100 VLANs across the street is necessarily a risk. Questions to ask and answer are what are the expected traffic patterns, both in terms of volume, and where are the endpoints of communication? Is there enough bandwidth and security inspection capacity to hairpin all traffic? If they are going to do firewall inspection and sniffing of traffic, can it operate at 10 Gbps? 20? how about 40 Gbps, or 100? What happens if the traffic exceeds inspection capacity? What traffic are they willing to be able to miss?
It is likely that core services such as DNS, authentication, 802.1x, Internet, etc. would be located at the older campus facilities. In such case, the new campus wouldn't be able to do much while separated from the older campus, so there isn't much point in trying to make it "survivable". In other words, the failure domain is likely to be unavoidably large. If they're planning on placing redundant data center resources in the new campus building, then having a separate failure domain makes more sense. Customer should also be looking at having second set of firewalls and other network inspection infrastructure.