Need Help Segregating Traffic Between 2 Companies via Layer 3 Switching

I currently have a big political problem; I currently have ten sites of which are utilizing frame relay in a hub and spoke fashion. The configuration was provided to me over two years ago (Thank LRMoore!)

                           Site 4         Site 5                                 Site 9
                                 \           /                                           |
                       384Kbps-FR  384Kbps-FR                      384Kbps-FR
                                     \    /                                              |
                                       \/                                                |
              FastEthernet----Site 1------------T1-P2P-----------Site 2----384Kbps-FR—Site 10
                     /                /|\                                              / | \  
                   /                /  |  \________1.54Mbps-FR___/   |   \
===ISP Wan Router   /     \                                              /       \
                                /         \                                          /           \
                    384Kbps-FR   384Kbps-FR               384Kbps-FR  384Kbps-FR
                             /               \                                   /                  \
                         Site 3          Site 6                        Site  7           Site 8

This frame relay solution is supporting 2 different organizations of which “had” a common interest. That common interest no longer exists and the 2 organizations are now splitting. We have an IT services agreement between my parent organization and the secondary organization because they are happy with the services we’ve provided them over the past two years. I wish we could just sign a business associate agreement between the two Org’s but its health care and HIPAA will not allow such a document to relinquish the need for segregated traffic for both organizations. They would like to continue the utilization of our network. The only way to accomplish this through my findings is through layer-3 switching (of which I have very little experience with). We are going to purchase a bunch of 3500 catalyst switch’s with EMI in order to allow L3 switching . Let me start off with some config backg ground at all of our sites:

I’ll start with the routing between the sites:

Currently we have an ip schema per the site:

Site 1 = 192.168.1.x
Site 2 = 192.168.2.x
Site 3 = 192.168.3.x
<etc>

Here’s an example config between site 1,2 and 3

Here's an example config for Site 1:

  interface FastEthernet 0
   ip address 192.168.1.1 255.255.255.0

  interface serial 0
   encapsulation frame-relay

!-- create sub-interfaces for each remote site
  interface serial 0.2 point-to-point
   description Site 2
   bandwidth 1544
   ip address 192.168.255.1 255.255.255.252
   frame-relay interface-dlci 200  <== DLCI # to be assigned by telco

  interface serial 0.3 point-to-point
   description Site 3
   bandwidth 384
   ip address 192.168.255.5 255.255.255.252
   frame-relay interface-dlci 30  

  interface serial 0.4 point-to-point
   description Site 4
   bandwidth 384
   ip address 192.168.255.9 255.255.255.252
   frame-relay interface-dlci 40

  interface serial 1
    service-module T1 clock source internal  
    description P2P to Site 2
    ip address 192.168.254.1 255.255.255.252

  ip route 0.0.0.0 0.0.0.0 192.168.1.2 <== Firewall/ISP WAN router
  router eigrp 101
   network 192.168.255.0
   network 192.168.254.0
   redist connected

SITE 2:
  interface FastEthernet 0
   ip address 192.168.2.1 255.255.255.0

  interface serial 0
   encapsulation frame-relay


  interface serial 0.1 point-to-point
   description Site 1
   bandwidth 1544
   ip address 192.168.255.2 255.255.255.252
   frame-relay interface-dlci 100  

  interface serial 0.7 point-to-point
   description Site 7
   bandwidth 384
   ip address 192.168.255.14 255.255.255.252
   frame-relay interface-dlci 70  

  interface serial 0.8 point-to-point
   description Site 8
   bandwidth 384
   ip address 192.168.255.18 255.255.255.252
   frame-relay interface-dlci 80

  interface serial 1
    description P2P to Site 2
    ip address 192.168.254.2 255.255.255.252

  ip route 0.0.0.0 0.0.0.0 192.168.255.1
  ip route 0.0.0.0 0.0.0.0 192.168.254.1  
  router eigrp 101
   network 192.168.255.0
   network 192.168.254.0
   redist connected


Site 3: (Sites 4-9 are virtually identical except for the IP addresses)
  interface FastEthernet 0
    ip address 192.168.3.1 255.255.255.0
    ip helper-address 192.168.1.255  
  interface serial 0
    service-module t1 timeslots 1-6  
    encapsulation frame-relay
  interface serial 0.1 point-to-point
    description Site 1
    ip address 192.168.255.6 255.255.255.252
    bandwidth 384
    frame-relay interface-dlci 100
  ip route 0.0.0.0 0.0.0.0 192.168.255.5
  router eigrp 101
    network 192.168.255.0
    redist connected



Currently the brunt of all of our windows based domain services and application services reside within the “Site 1” location. We have a domain controller located at every site because we run file replication services at every site in order to allow user’s access to their files (in a roaming user profile) when roaming from site to site.

The server’s ip’s located on site 1 are as follows:

Exchange Server: 192.168.1.249
Domain Controller: 192.168.1.250
SMS Server: 192.168.1.248
Other Server’s: 192.168.1.24x – 192.168.1.25x

The server’s ip’s located on site 2 are as follows:
Domain Controller: 192.168.2.250

The server’s ip’s located on site 3 are as follows:
Domain Controller: 192.168.3.250

etc.

We only have domain controllers and workstations outside of site 1; all application service provision is provided via site 1.


The situation that we’re in is that we need to separate the network logically, not physically. We’re currently using cheap dell switch’s of which offer us no QOS and no intelligent management of traffic. We share office space at every site so we will need to implement segregation at every site. I need to incorporate OSPF for high-speed convergence in the case of failure, vlan’ing or VACL if possible and I also need to provide QOS on the switch for services. I also need to separate the ip schema’s for all sites:

i.e. keep the current schema for company A 192.168.1.x and 172.19.1.x for the other sites. The other complication is that the users in company B will need to communicate with the servers of which belong to company A. All of the servers are owned by company A. Can someone please help me with the configuration of said switch’s. I don’t know where to begin and where to end. I know you guys can help. Thanks
TheLankAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

SorensonCommented:
I am not sure you need the 3550 switches with EMI on them at this point.

If I understand your drawing and explanation correctly, you need to seperate traffic at each site so that 2 pcs, next to each other, from seperate companies, do not see each other.  

I would start by placing vlans on the switches at the remote sites, and changing the fast ethernet ports on the routers to dot1q trunking ports.  Having cisco switches at the remote ends will certainly help, but if the routers are 26xx, 28xx, or 36xx, 38xx you should be able to use them to route the vlans, and not need an L3 switch (at least not at the remote sites).  This will make you create new subnets for company B at each location.

Once the vlans are in place at the remote sites, and the routers are trunked, policy based routing PBR can be used on each vlan subinterface to keep traffic flowing in the right direction and away from each other.  The PBR for the remote sites, would push ip for the local server to it, but redirect other ip traffic back towards the core routers, those routers would need pbr to keep the traffic seperated and flowing towards the centralized servers or the internet.

I am not sure I would use OSPF, especially if you are going to be using cisco equipment.  Most of the routing would be handled by the PBR instead of a route table anyways (to keep items secure and away from each other).

If I am heading in the wrong direction, let me know.  If it is right we can get into some sample configs to control the traffic.
0
TheLankAuthor Commented:
Hey, Thanks for the fast reply!!!! Your in the right direction; in that case I'll not be ordering 3500 series cat's; instead I'll get 2960's. Let me give you a deeper view of the network architecture:

                           Site 4         Site 5                                 Site 9
                                 \           /                                           |
                       384Kbps-FR  384Kbps-FR                      384Kbps-FR
                                     \    /                                              |
                                       \/                                                |
          FastEthernet----Site 1------------T1-P2P-----------Site 2----384Kbps-FR—Site 10
                     /                /|\                                              / | \  
                   /                /  |  \________1.54Mbps-FR___/   |   \
===ISP Wan Router   /     \                                              /       \
                                /         \                                          /           \
                    384Kbps-FR   384Kbps-FR               384Kbps-FR  384Kbps-FR
                             /               \                                   /                  \
                         Site 3          Site 6                        Site  7           Site 8

We're currently utilizing 2 cisco 2610's at the site1/site2 distribution points and utilizing 1721's at the frame leg's. We have dell 2324's (unmanaged) behind each 2610/1721. The dell's were on the cheap and offered 2 1gb uplinks for interconnectivity between switches. The only site of which containts different switches is site 1; not only do we have the dell's for workstations connectivity but we feed the uplinks of the dell's into 2 belkin 10/100/1000 24 port (unmanaged) switches to give our users redundancy (every server has two nics; one nic going into one belkin and the other nic going into the other belkin.) I'd like to mirror this but with the managed layer 2 solution. In this case we'd be using 17 WS-C2960-24TT-L's (24 ports 10/100 2 ports 1000 uplink) for workstation/router connectivity amongst the sites and 2 WS-C2960G-24TC-L's (24 ports 10/100/1000) for the server redundancy over at site 1. And again, we still need company B to see our servers. Thanks again!
0
SorensonCommented:
The 2610s will not support the 802.1q intervlan routing, but the 1721s will.  In general you need a 100mb ethernet interface or better to suppor the 802.1q trunking.  However your config shows a fastethernet port on the site 1 router.  This page indicates it would work, but I am not sure:  http://www.cisco.com/en/US/products/hw/routers/ps259/prod_bulletin09186a00800921e4.html

So as a sample for the 2960 switches

!
vlan 2
 name BusinessB
!
! configure ports for business B
int range fa0/x -xx
  switch mode acces
 spanning-tree portfast
  switch access vlan 2
!
! remaining ports stay in vlan 1 which will remain business A
int vlan 1


0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

SorensonCommented:
sorry, got cut off

!
int vlan 1
  ip addr x.x.x.x y.y.y.y
!
!next setup trunking interface to router
!
interface fa0/24
  description link to 1721 router
  switch mode trunk
  switch trunk encap dot1q (not sure if 2960 requires it, as some 29xx only support 802.1q)
  switch mode trunk
!

on the 1721 router
!
 interface FastEthernet 0
   no ip addr
!
   interface FastEthernet 0/0.1  
     encap dot1q 1 native
      ip addr 192.168.3.1 255.255.255.0
      no shut
!
   interface FastEthernet 0/0.2  
     encap dot1q 2
     ip addr 192.168.30.1 255.255.255.0 (new ip address for business B gateway)
     no shut
!    


also see: http://www.cisco.com/warp/public/473/50.shtml
0
SorensonCommented:
once the basic mechanics are in place, then you can start restricting who sees what / where...
0
TheLankAuthor Commented:
Damn, the config was taken from a document I wrote some time back. The actual only differs in actual interface is ethernet 10bT. I can upgrade the two sides to 2621's with dual fe ports 16/64. After inserting said config on the hardware side how would I specify via PBR through config as to what access levels are given; can you give me samples?
0
SorensonCommented:
If you are going to upgrade, it gives you a perfect time to test the configs before throwing them in.

For the PBR, will they need access to the a server in the remote site, as well as access to the main site?
0
TheLankAuthor Commented:
That's the complicated part; the domain controllers (10 in all) need to speak to one another. The answer to your question is yes; the clients located at the remote sites will need access to the server local to them and the main host site.
0
SorensonCommented:
Assuming that AD sites / subnets are all configured and updated after the new subnets and vlans are put into place....

remote site:   192.168.3.x business A local lan, 192.168.30.x business B local lan, 192.168.3.10 localserver 192.168.2.10 site1server

remote site router
!
access-list 100 remark ControlAccess
access-list 100 permit ip 192.168.30.0 0.0.0.255 host 192.168.3.10
access-list 100 deny ip 192.168.30.0 0.0.0.255 192.168.3.0 0.0.0.255
access-list 100 permit ip 192.168.30.0 0.0.0.255 host 192.168.2.10
access-list 100 deny ip 192.168.30.0 0.0.0.255 192.168.2.0 0.0.0.255
!
access-list 120 remark Force-Route
access-list 120 deny ip any host 192.168.3.10  (needed to keep packet from going to main site)
access-list 120 permit ip any any   (all other packets go to main site)
!
route-map RouteFix permit 10
  match ip address 120
  set ip next-hop 192.168.255.5
  set ip default next-hop 192.168.255.5
!
int fa0/0.2
!
int fa0/0.2
   ip access-group 100 in
   ip policy route-map RouteFix
!

Route maps would continue to be needed to force the packet along its way, allowing it to drop out for server access, or pushing its next hop to the internet router.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
TheLankAuthor Commented:
Thanks for the sample; i'm a little confused with the following line

access-list 120 deny ip any host 192.168.3.10  (3.10 representing the domain controller in the sample config)

I need the servers at all sites to be able to see one another. It the workstations at every site that the concern lies with. As was discussed before, I can not have company a workstations see company B workstations. Basically administrators/servers and/or endpoint devices will need to see both companies. Also, I will be linking up switches via the GE 0/1 and GE 0/2. I will probably need to carry over trunk information between 2 switches. Can you provide a sample for the trunk carry over between 2 2960?
0
SorensonCommented:
The line will force the packet out of the PBR (policy based routing) and therefore it will follow standard routing (not get the next hop set), to get to the correct host.  Any traffic that "matches" or is permitted by the ACL is forced to next hop back to the main site, which would force it out to the internet with a similar PBR.  The deny allows traffic to drop out of the forced path and follow the routing tables to get to the host.
0
SorensonCommented:
If the line was removed, traffic to that server, even though the server is connected on the local lan, would be forced back to the main site.
0
TheLankAuthor Commented:
Thanks for the follow up and thanks again for the help!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Routers

From novice to tech pro — start learning today.