I am pretty sure I have this concept 95% complete, but wanted to run it by others to see what you suggest or different ideas.
So basically we currently have a location that has a router and a switch at the head end. I bring in the ISP connection to the switch on a vlan and pass that along to the router for it's role. Router on a stick. The reason I do it this way is because this site is a multi vendor building and we have multiple businesses using public IP's from us for their firewalls etc. Which is why I brought the IP's in via the switch so I can pass them down the line via layer 2. Works great.
Now we are going to be bringing in a larger pipe from a separate provider for multi-homing and I am certain the way I need to do this is the same thought. Bring it in on its own vlan and work on changing over the other businesses one at a time because of down time reasons. I do not think it is possible to do the switch all at once. I am sure this will work as well and will have not problems.
Here is where I need some advice, after I switch over all the public IP's, I need to setup failover. How would I accomplish this with a cisco 2921 for instance in this setup? I do not have the ability to use BGP for both providers, only one, so not help there. Is there a standby shut I can put on the vlan if port X goes down, the vlan activates? How could I get other devices to do the same? As long as it is supported of course. I know one device is a cisco 2901.
I hope this helps