Solved

Access list blocking communication on ASA.

Posted on 2013-05-28
17
745 Views
Last Modified: 2013-06-15
Hello

I would like some help  in locating access list lines on the ASA.
Recently, after a switch change, the monitoring  network lost access to  one of our  management networks. The funny thing is that the monitoring network that comes to the ASA, can reach the ASA perfectly, and the monitored network can reach the gateway ( which is the ASA), without any issue.

eg. The ASA can ping both ends without any issue but both ends cant ping each other, and the ASA is the  being the break point.

Both networks reach the ASA
Packets are arriving to the ASA (checked with capture )

I did a packet tracer  from  end to end, and  according to it the traffic from the monitoring network  network  to the  management network should work fine. And in the capture i can see the echo request.

 However the way back shows me a drop  due to access list. And of course i cant see the echo reply
 I tried locating that access-list id  on the configuration but I just cant find it!!!!
The id showed on the packet tracer is not shown on the show run command.

Please if someone cud help giving a clue on how to get the line that blocks traffic it will be great. So far i just have the id number for the ACL and  that number is not in the access lists applied to the interfaces.



Thanks a lot !!!
0
Comment
Question by:kumo_chan
  • 9
  • 8
17 Comments
 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39200954
first clear the counters

clear access-list acl_name_here counters

then run your test and look at the counters

sh access-list acl_name_here


chances are that a >0 count will be seen on more than just the entry that is affecting your traffic but it should hopefully at least narrow it down.
0
 

Author Comment

by:kumo_chan
ID: 39206404
Hello Cyclops,

Thanks for your answer, I tried that but the actual counters that increase are the permit. I am quite new to the ASA, and I have been checking, I  think the issue is that the Security Level among the interfaces is different but I am not sure. Specially because some IPs within the same network  actually work.

For example, from my office PC i can ping the 10.2.1.2 but not the 10.2.1.4 ? however the gateway (the asa), can ping everything.

Could you please take a look at my packet tracer not sure if it will help at all?
 and Is there any chance I can apply an access list to allow this traffic ?  my ASA version is 8.2.

Thanks a lot!!!
0
 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39206620
the security level is supposed to be different so that is fine.

i'm not seeing any packet tracer anywhere in the post or an uploaded file.  Am I missing something?

yes, you can absolutely apply an access list to allow this traffic.  Since you are new to ASA's, I would recommend using the ASDM then for looking at this issue.  

Can you post the source-destination IPs and the ACL being implemented that you are using.  I should be able to help from there.  The other thing you can do though is that when you are seeing the logs scroll by in the ASDM tool you can click on the entry stating the denied packet and from there you should be able to right click or something and have it direct you to the ACL entry that is denying the packet also (I haven't done it yet before though but read it is possible) you are supposed to be able to create a new rule that would allow the denied traffic
0
 

Author Comment

by:kumo_chan
ID: 39206984
Can you see it now?  I am new in these blogs so I can see it but I think it had to do with the settings.
comm.txt
0
 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39207341
first, i'm assuming that ACL you posted is applied to the management interface in the inbound direction.

second, do you have a route for the 10.4.2.0/24 network since its wanting to go out the office interface for some reason.  It may be failing because it doesn't know how to forward the traffic appropriately.
0
 

Author Comment

by:kumo_chan
ID: 39207437
Hello Cyclops!,


Well The ACL is applied to the office interface, not to the management, I have no access lists applied to the management interface so far.
 
The office network (10.4.2.0/24) has a  connection to the  ASA through interfaceGigabitEthernet0/3.701, the office router basically forwards traffic to this network to the ASA in this interface for the management.

I do have a route for 10.4.2.0 through interface 3.701.
and I  even have a route for the management network (through the management interface GigabitEthernet0/3.54),  as well even though it is a direct connection to the ASA.

but still what I cant comprehend is why  my office can ping a few devices  in the managemente network but not all  and still the ASA can reach all of them.

I have been doing packet captures as well and still if I ping I can see only the request, no reply from  a few devices from the management network to my office network. But all respond to the ASA.

Do you by any chance know any way to know if the ASA is actually sending the packet (which is not packet capture because it doesnt appear), I am beginning to think that maybe  the devices are the ones not responding, or they are not getting the icmp replies for some reason....

Thanks  again!!!!
Regards,
0
 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39207473
ok, that's why then.

management - security level 70
office - security level 85

going from lower security level (less trusted) to higher security level (more trusted) is by default denied.  You have to create an acl that allows the traffic you want and apply it in the inbound direction on the management interface.  That will fix your issue (unless there is something else I'm not aware of but this for sure is blocking it right now).

the asa can reach all of them because there are parts of the packet life cycle that are skipped due to the asa being the source.  as to why the office can ping some and not others, I'm not seeing a packet trace for the echo reply from mgmt to office for the host being pinged.  since there is no acl on the mgmt interface, my guess is that you might have icmp inspection turned on but would need to see your class-map, policy map, and service policy config.  however your packet trace shows starting from a mgmt side host so due to security levels is guaranteed to be firewalled.

you can run a capture actually on the asa.  you just create an acl first though otherwise too much to look thru that targets your test.  apply it with a capture command, run the test, and then look at the results with show capture.

however first you may want to add an acl to the mgmt int to allow the needed traffic.  if not mgmt hosts need to initiate traffic to the office hosts and you're just talking about pinging then please make sure "inspect icmp" is included in the applied service policy.  if you post that relevant config I can help you get it added.
0
 

Author Comment

by:kumo_chan
ID: 39207585
Hello Cyclops!!!,

Please find bellow the configuration.  A little simple hehe.

class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map
  inspect ftp
  inspect h323 h225
  inspect h323 ras
  inspect rsh
  inspect rtsp
  inspect esmtp
  inspect sqlnet
  inspect skinny  
  inspect sunrpc
  inspect xdmcp
  inspect sip  
  inspect netbios
  inspect tftp
  inspect icmp
  inspect ipsec-pass-thru
!
service-policy global_policy global
prompt hostname context

Thank you soo much!!!

Kumo..
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39207683
you already have icmp inspection so return echo replies would be allowed for office side hosts pinging mgmt side hosts.

can you run a packet trace for the host on the office side to the mgmt side that is failing?  it should be getting to the device and as long as an echo reply is being sent it should return.

also, while you're at it, run a capture from the asa itself

Capture examples:  http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080a9edd6.shtml
0
 

Author Comment

by:kumo_chan
ID: 39207766
Hello Cyclops,

I tried that packet capture before and from the office to the management it works (at least on the packet tracer).  

I also did a capture and I  still can see only requests, not replys, what I am not sure is if is the ASA the one that is actually blocking, or is the device not responding for some reason.

I previously did wireshark on the ASA port for the management VLAN, but I can see exactly the same as the packet capture, just requests but no replies from the 10.2.1.4, unless the ASA itself pings this host.

Please find attached the file with the info... not really sure if my packet capture  is what you asked?

Thanks!!!!

Regards,
comm2.txt
0
 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39207800
based on that I'm thinking either one of two things.

1) the host being pinged has a firewall that limits what is allowed to ping it.
the ASA just happens to be in that list
2) the destination host isn't configured with an appropriate default gateway.
the ASA would be using the mgmt IP address to ping the host so since it is on the same subnet, no routing (hence no default gateway usage necessary) happens

can you log onto the 10.2.1.4 host and ping an office host.  we know that it'll be dropped by the asa, but because there is no echo-reply even being seen by the asa I'm leaning toward this not being an asa problem.  the asa looks to be configured just fine.  I think its a host misconfiguration issue.  have you ran a capture from the mgmt host?  what kind of host are you pinging at 10.2.1.4?
0
 

Author Comment

by:kumo_chan
ID: 39207976
Hello Cyclops,

Thanks again :D !!

I can do that tomorrow, the 10.2.1.4 is managed by a vendor it has RIPv2 and is the management network  for a  few Microwaves and  I will need those details  so most probably will be posting more details tomorrow.


I will be back!!!

Thanks!!!!
0
 

Author Comment

by:kumo_chan
ID: 39212411
Hello Cyclops,

Well something really weird happened after the tests, I did I diagram to see if it helps u to understand. frankly after the test I got even more confused.

On the Diagram both vlans come to one interface on the ASA (as a subinterface for their vlan of course). and  the ASA is the gateway.
 then IPS 10.2.1.3, 10.2.1.4 and 10.2.1.2 are all coming from the same interface into my switch.

I went to the other extreme, where my managed devices are and did the packet capture with wireshark, I am able to see the requests but not reply, so I thought is the managed device it is not answering for some reason that is the issue.  Then in order to  prove this I connected my computer  to the same interface, pinged the gateway successfullyperfect! , and then when I tried to ping  my laptop from the office nothing happened. IT is just the same thing that with the managed device, I can reach the ASA, but cant reach the Office network.

Wireshark shows me all the packets coming to that vlan but is like if the PC doesnt recognize it is for it, I dont send replies, only to the gateway. I can just see the requests coming.

 Also,  I was able to ping 10.2.1.2 and 10.2.1.3 from the office before, but when I unplugged this connection  in order to test with my PC and then put the original cable back into my switch I lost all communication with that network.  so basically i can ping from the  ASA to these addresses but thats it, nothing more.

This just got me even more confused.

I hope this makes  some sense for u, cause it  is just killing me at the moment hehe.

Thanks again!!!
Regards,
2013-06-01-14.53.46.jpg
0
 
LVL 25

Expert Comment

by:Cyclops3590
ID: 39212984
sorry, more questions now.

1) they are different vlans off the same physical interface of the asa but your diagram shows them going into different devices.  Can you supply a physical diagram as well as a logical diagram?
2) you say the mgmt network hosts are off the same interface of the switch.  can you elaborate how that is so?
3) did you verify the host default gateway configuration?  because the host is on the mgmt network it'll be able to ping the asa interface.  look at the route table and verify that for the office network it has  route back.
4) when you connected your laptop, did you run wireshark from it and at least see a echo reply being sent?  I understand the asa might not have seen one but we need to verify that the managed devices are actually sending them or not.  so if the laptop is sending on and the asa doesn't see one then something in between in the physical flow is causing it to be dropped.
0
 

Author Comment

by:kumo_chan
ID: 39214146
Dont worry cyclops, I really appreciate the help ! feel free to ask.
1. Diagramas are attached.
2. These three devices I have there are the management ports for a wireless link,  and each of them has a small 4 port  hub  included as part of the hardware.  so one port is going to my switch, and the others are daisy chained to each other. I hope the diagrams can make  more sense for you?
3. Host  default gateway was correct, and they can ping the gateway as well.  both networks are reached from the ASA through an static route.
4. This was the weirdest of all the tests. I did put wireshark on my laptop and connected straight into the port  that goes to my switch, I  was able to see all the incoming packets but never saw a reply from my laptop. as if for some reason my laptop wasnt replying.
It had the correct gateway though and it  was able to ping it. It will just sort of ignore everything is not coming from the gateway.

If i connect to  the managed devices and run wireshark from them is just exactly the same, packets are coming but no reply. I thought it was a failure with the devices, but I know my laptop works fine and couldnt get any reply  from it on thw wireshark capture either.

 Thanks again!!! I really really appreciate the help!

Regards,
Logical.jpg
physical.jpg
0
 
LVL 25

Accepted Solution

by:
Cyclops3590 earned 500 total points
ID: 39214357
can you elaborate a little more on what exactly the "switch change" was?  which switch was changed? what was the change? is the config identical?  what was the reason for the change?

this thread is getting a little long so I want to try to recap to make sure we're on the same page and not missing anything.


Problem

Monitoring within the office network used to be able to reach the 10.2.1.x management networking before a switch was changed.  After the switch change, it appears that management devices don't even respond to office traffic.  However the ASA, which is the gateway to the mgmt network, can easily communicate with everyone.

Troubleshooting thus far

Performed a capture on the office side - see echo but no echo reply
performed capture of office client at asa - same as above
performed capture on mgmt side of office client - same as above
performed capture on laptop on mgmt side of office client - same as above.  laptop didn't even see an echo reply from itself.
ping from asa to mgmt device - works
ping from anything behind asa on office side to mgmt device - fails
ping from office device to asa  - works
verified mgmt devices have default gateway (or default route) to asa

Learned so far

Layer 1 - clearly fine
Layer 2 - fine because of pings to/from asa.  switches should be configured right with vlan tagging and such otherwise asa shouldn't be able to ping it as mgmt is vlan 54 tagged and default vlan config is 1 untagged.
Layer 3 - supposedly configured correctly but shows symptoms of mgmt devices not understanding how to get off of own subnet (just seems like no route to get to 10.4.1.x/24)


to be honest it really concerns me that when you plug in the laptop into that mgmt port, assign it the same IP as one of the mgmt devices and try to ping it that it sees the echo but doesn't send an echo reply.  even if the switch is messed up, the laptop itself (provided it has a route for the source device) should still send an echo reply and see it is sending that echo reply.  The only way I know it wouldn't is if it doesn't have routing for the IP it received the echo from, in which case it would drop an reply as its unreachable.  the asa doesn't do some funky nat'ing does it?  Does the mgmt devices have a default route, default gateway, or specific route configured?  Can you do a capture at the asa and from your laptop or the mgmt device and do a ping from the mgmt device to the office device?  Although the asa should drop the packet I want to see if it even generates an echo.  If it doesn't then it lends more credence to being a routing issue, but I'm not sure how since the only thing that changed is the switch, right?
0
 

Author Closing Comment

by:kumo_chan
ID: 39250912
Hello Cyclops,

You were totally right, it was an issue with the managed devices,  i had to wait for the vendor to come in and check the devices himself, he had some routing configured within the managed devices which was causing an issue. Even though he assured there was a gateway and no routing, when he came and we both entered to the devices we found the following:

1. Rip was configured in a wrong way,
2. there was no gateway  in any of them.
 3. routing tables were sending traffic somewhere else

Therefore i assume that  the old switches with the old EXtreme IOS 12.2.x  had a bug that somehow made that setup to work all these years, and when we introduced the new EXOS 15.x.x  it actually worked as it was supposed to?. Or maybe with the changes the managed devices got blocked ??? I have no idea, but i really appreciate your help !!!

Thanks a lot !!!
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Join & Write a Comment

Suggested Solutions

I have seen some questions on problems with SSH/telnet access to Cisco routers that may occur despite the fact that from a PC connected to your LAN, Internet connectivity is in place and users can access Internet sites without any issues.  There are…
There are two basic ways to configure a static route for Cisco IOS devices. I've written this article to highlight a case study comparing the configuration of a static route using the next-hop IP and the configuration of a static route using an outg…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now