Problem with no Internet - I fixed problem, but not sure what I on

OK Cisco Gurus, can you help me with a question.  I had a problem yesterday with my Internet connection going down and all my VPNs could not send me data.  I got it going late last night, but I want to know why….read on.

I have a Cisco 515 as the center spoke to my network.  I have one remote office connected over a VPN through an E1 (Cisco 501).  I also have 10 remote sites (and many more on the way) that have DSL and connect to me over VPNs via Linksys RV042s.  I have been installing the RV042s over the last week or so, adding one or two a day.

Yesterday morning, my E1 (501) site called me to say their Internet was very slow.  Then they lost browsing, but messenger still worked.  Eventually they lost all Internet connectivity, including our VPN link.  I still had access to this site and the Cisco 501 over a RSH connection, but eventually I lost this connection also.

As I was tying to figure out what happened to my connection to my remote 501, I lost Internet at my main site.  I had a laptop connected outside my Firewall (515) with a public IP and it still had Internet, so I did not lose Internet from my T1 provider, I just could not get anything out or in from my private LAN.  I could not even ping my laptop from my 515 though they were on the same segment.  I had not changed anything since the day before, so I was baffled.  Besides I just added a few lines of code for NAT avoidance and a Peer Address for my Crypto Maps.  Also, the one remote RV042 that I allow Internet Access to (port 80) also lost Internet connection.  All the other sites have Port 80 blocked.

I reviewed the code over and over, line by line and I could not figure out what happened.    I checked my ISAKMP SAs, and my remote VPNs from my RV042s were still connecting, but I could not send them data.  I tried reloading, clearing connections, clearing IPSEC and ISAKMP SAs, etc.  It was like my default gateway went away, but I verified it was still there.

I decided to copy the config from a text file I had saved from 6 months ago, before I began experimenting with VPNs, and paste the lines into my Telnet session to my 515.  Note, I did not return my 515 to factory just yet; I just copied the lines from my archived Text File and paste them to my current Telnet Session.  The second I did this, my Internet came up.  I cleaned up the config and made sure it was current.  Again, all my previous lines for my new VPNs and my Access Lists were still there from before.  Anyway, I could now communicate with all my RV042s over their VPNs.  

Shortly thereafter, I could RSH back to my Mexico site.  After clearing my IPSEC SAs and ISAKMP SAs, that VPN came back up.  All is well now.  I found out my provider to my remote E1 (501) had a problem (thought they won’t admit it) with their E1 feed to me.  In any case, all is working well now, but I just want to know what I did, and if there is something else I could do.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Have you checked the logs on all the devices?  Do you have a syslog server that accumulates events you could check?
From the PIX to the E1, how are you connecting? Did you have any interface errors on either inside or outside interfaces? What happens with these devices are; when the interface errors increase, it will just go down one day and a simple reset will bring back everything up and fine. It is a known thing.

There is still time, can you see if your interfaces on PIX have any errors (By errors I mean data errors).

You said you copy an archived config to your new config without resetting it to factory default? So that would mean any duplicates will just be ignored with a warning message from the PIX and things like access-group, crypto map applied on the interface will be overwritten. What could be the key to this issue is the changes you were making on the VPN specifically the crypto maps and the difference of the old config and new config.

>>>> Besides I just added a few lines of code for NAT avoidance and a Peer Address for my Crypto Maps

Exactly what commands were you adding?

I can guarantee you this: an incomplete crypto map will cause the internet traffic to halt and prevent communication
even to the interface itself where the crypto map is applied on"

Here's a scenario, let's say you have the following VPN config:

crypto map test 20 match address 100
crypto map test 20 set transform-set 3DES
crypto map test 20 set peer
crypto map test interface outside

You decided to add a new one but did not add the match address right away (because you had to answer a call = ) )

crypto map test 20 ipsec-isakmp
crypto map test 20 set transform-set 3DES
crypto map test 20 set peer

Since the crypto map is already applied because of an existing configuration, the addition of the above configuration will
cause the PIX to lose not only its internet connection but the ability of other to host to communicate to its outside interface (which is why you can't ping it from your laptop)

Why it does that you may ask? Because with a missing match address on one of its crypto map the PIX will try to encrypt all traffic and anything unencrypted is drop.

I'm not sure if this is what you have run into but this is one of the disastrous mistakes I have encounter with my customers which I am sharing to everybody.

Javier196Author Commented:
The problem occured out of the blue.  The day before, I added a crypto line to add a peer, and I added a line for a isammp key for a new VPN device.  The next morning, all was fine.   Everyting was connected and my VPNs were up and communicating.  About 10:00, one of my sites (Cisco 501) started complaining about not being able to access some Internet sites.  They then only had messenger connected.  They eventually lost alll Internet aceces.  Shortly thereafter, I started having problem with my firewall at my site, and my Internet Access went down.  I did not have any tools to tell me what was going on, so I just kept checking all the config lines.

I found out the next day that the remote site that had the original Internet issues, had a problem with their E1 and may have caused the whole issue.  After I got my site back up and running, their site came back up and things have been fine since.  I have not changed the config.  I just copied a older config that did not have the new VPN lines onto our Firewall via HyperTerminal.  Again, things have been fine since, and this problem occured over 3 weeks ago.
PAQed with points refunded (500)

Community Support Moderator

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.