BGP issue - backup circuit utilization when it shouldn't be...

Have a cisco 3750 running BGP across two circuits.  Primary is a 100meg Ethernet from the main ISP, and the secondary is an Ethernet service provided by the data center we use.  The secondary is billed for utilization only, and we never had a utilization bill, until last month.  Seems we are using traffic across the backup circuit, even though the bgp log on the router doesn't show the primary being down.  The last event shows: Last reset 3d22h, due to Interface flap - and usage on the secondary even today.  

Unfortunately I'm no expert on BGP and was hoping for some suggestions on what to look for.
mchad65Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jan SpringerCommented:
A couple of things:

# show ip bgp neighbor <neighbor ip> advertised-routes

for both.  are they any advertisements of equal weight that would cause traffic to swing to the backup when the primary interface flapped?  if so, reset the backup interface to force traffic back over the primary.
0
mchad65Author Commented:
Here's what I get:

For security, I masked the ip's as follows.  I left the fourth octet exposed.

Primary ISP IP=<primary neighbor.149>
Secondary ISP IP=<secondary neighbor.121>

Router#sh ip bgp neighbor <primary neighbor.149> advertised-routes
BGP table version is 137488, local router ID is <secondary neighbor.122>
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network                                             Next Hop                               Metric LocPrf Weight Path
*> 0.0.0.0                                              <primary neighbor.149>                      150           0 6128 i
*> <primary neighbor.148>/30         0.0.0.0                                              0              32768       ?
*> Our network/24                              0.0.0.0                                              0             32768        ?
*> <Secondary Neighbor.120>/30
                                                                0.0.0.0                                              0             32768        ?

Total number of prefixes 4


SirotaEdge#sh ip bgp neighbor <secondary neighbor.121> advertised-routes
BGP table version is 137488, local router ID is <Secondary Neighbor.122>
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network                                             Next Hop                               Metric LocPrf Weight Path
*> 0.0.0.0                                              <primary neighbor.149>                      150           0 6128 i
*> <primary neighbor.148>/30         0.0.0.0                                              0              32768       ?
*> Our network/24                              0.0.0.0                                              0             32768        ?
*> <Secondary Neighbor.120>/30
                                                                0.0.0.0                                              0             32768        ?

Total number of prefixes 4
Total number of prefixes 4

The Local router IP on both shows as what looks like the next hop on the secondary ISP.  That doesn't look right...
0
Jan SpringerCommented:
So, your /24 is equally preferred through both providers.  Are you using anything like AS-Path prepends to prefer your primary provider in the event of a failed link (whether a flap or an outage) upon return of service?

Are you accepting a default route from both providers or have you configured weighted default routes manually?
0
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

mchad65Author Commented:
Thanks Jan.  I have to admit, the BGP config was built by a contractor.  I'm trying to avoid calling him in.  BGP config is not my area of expertise.  That being said, if you can give me the commands to check, I can.  Otherwise maybe its best I call him in.  I only posted here because this has been set up for several years now and this is the first time I'm seeing utilization on the secondary circuit.
0
Jan SpringerCommented:
sh run | inc ip route 0.0.0.0

sh ip bgp neighbor <neighbor IP> received-routes | inc 0.0.0.0


copy and paste those commands.  the pipe ( | ) requires a space before and after.

do a "sho run" and look for a policy-map that defines as-path prepends.  if you want to inbox me here at EE with your prefix, I can privately reply back with what i see.
0
mchad65Author Commented:
Hmm.  Neither commands return a response from the enable prompt provide any response.

Sh run has no policy-map whatsoever.  All I see relevant is:

router bgp <masked>
 no synchronization
 bgp log-neighbor-changes
 timers bgp 15 45
 redistribute connected
 neighbor <primary neighbor IP>  remote-as <masked>
 neighbor <primary neighbor IP> description BGP with <primary ISP> via metro ethernet
 neighbor <primary neighbor IP> route-map setlocalpref in
 neighbor <secondary neighbor IP> remote-as <masked>
 neighbor <secondary neighbor IP> description BGP with <secondary ISP> via data center backbone
 no auto-summary

route-map setlocalpref permit 10
 set local-preference 150

Nothing sensitive here, yet...
0
Jan SpringerCommented:
I'm not seeing as-path prepends with a route-map statement applied to your backup provider.

that's what you need:

route-map BACKUP permit 10
 set as-path prepend <yourASN> <yourASN> <yourASN>

router bgp <yourASN>
  neighbor <backupIP> route-map BACKUP out
0
mchad65Author Commented:
Thanks for your help!

Just to clarify:  set as-path prepend <yourASN> <yourASN> <yourASN>  Where "<yourASN>" is repeated three times?
0
Jan SpringerCommented:
Yes, just in case your backup provider has upstream connections that make it preferred.

You can start with one, then add a second if you want.  It's just easiest to do 3 and call it a day.
0
mchad65Author Commented:
Ok.  I'll need to do a little research on these commands beforehand, since I am not familiar.  No offence whatsoever - not suggesting you are trying to blow up my network :-)

Only that this is our corporate perimeter router which I basically never touch, and I don't want to screw anything up in case I wasn't perfectly clear on anything.  Due diligence and all!  

Thanks very much for the assist and I'll post the results.
0
Jan SpringerCommented:
You're welcome and no offense taken.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Routers

From novice to tech pro — start learning today.