Access list blocking communication on ASA.
Hello
I would like some help in locating access list lines on the ASA.
Recently, after a switch change, the monitoring network lost access to one of our management networks. The funny thing is that the monitoring network that comes to the ASA, can reach the ASA perfectly, and the monitored network can reach the gateway ( which is the ASA), without any issue.
eg. The ASA can ping both ends without any issue but both ends cant ping each other, and the ASA is the being the break point.
Both networks reach the ASA
Packets are arriving to the ASA (checked with capture )
I did a packet tracer from end to end, and according to it the traffic from the monitoring network network to the management network should work fine. And in the capture i can see the echo request.
However the way back shows me a drop due to access list. And of course i cant see the echo reply
I tried locating that access-list id on the configuration but I just cant find it!!!!
The id showed on the packet tracer is not shown on the show run command.
Please if someone cud help giving a clue on how to get the line that blocks traffic it will be great. So far i just have the id number for the ACL and that number is not in the access lists applied to the interfaces.
Thanks a lot !!!
I would like some help in locating access list lines on the ASA.
Recently, after a switch change, the monitoring network lost access to one of our management networks. The funny thing is that the monitoring network that comes to the ASA, can reach the ASA perfectly, and the monitored network can reach the gateway ( which is the ASA), without any issue.
eg. The ASA can ping both ends without any issue but both ends cant ping each other, and the ASA is the being the break point.
Both networks reach the ASA
Packets are arriving to the ASA (checked with capture )
I did a packet tracer from end to end, and according to it the traffic from the monitoring network network to the management network should work fine. And in the capture i can see the echo request.
However the way back shows me a drop due to access list. And of course i cant see the echo reply
I tried locating that access-list id on the configuration but I just cant find it!!!!
The id showed on the packet tracer is not shown on the show run command.
Please if someone cud help giving a clue on how to get the line that blocks traffic it will be great. So far i just have the id number for the ACL and that number is not in the access lists applied to the interfaces.
Thanks a lot !!!
ASKER
Hello Cyclops,
Thanks for your answer, I tried that but the actual counters that increase are the permit. I am quite new to the ASA, and I have been checking, I think the issue is that the Security Level among the interfaces is different but I am not sure. Specially because some IPs within the same network actually work.
For example, from my office PC i can ping the 10.2.1.2 but not the 10.2.1.4 ? however the gateway (the asa), can ping everything.
Could you please take a look at my packet tracer not sure if it will help at all?
and Is there any chance I can apply an access list to allow this traffic ? my ASA version is 8.2.
Thanks a lot!!!
Thanks for your answer, I tried that but the actual counters that increase are the permit. I am quite new to the ASA, and I have been checking, I think the issue is that the Security Level among the interfaces is different but I am not sure. Specially because some IPs within the same network actually work.
For example, from my office PC i can ping the 10.2.1.2 but not the 10.2.1.4 ? however the gateway (the asa), can ping everything.
Could you please take a look at my packet tracer not sure if it will help at all?
and Is there any chance I can apply an access list to allow this traffic ? my ASA version is 8.2.
Thanks a lot!!!
the security level is supposed to be different so that is fine.
i'm not seeing any packet tracer anywhere in the post or an uploaded file. Am I missing something?
yes, you can absolutely apply an access list to allow this traffic. Since you are new to ASA's, I would recommend using the ASDM then for looking at this issue.
Can you post the source-destination IPs and the ACL being implemented that you are using. I should be able to help from there. The other thing you can do though is that when you are seeing the logs scroll by in the ASDM tool you can click on the entry stating the denied packet and from there you should be able to right click or something and have it direct you to the ACL entry that is denying the packet also (I haven't done it yet before though but read it is possible) you are supposed to be able to create a new rule that would allow the denied traffic
i'm not seeing any packet tracer anywhere in the post or an uploaded file. Am I missing something?
yes, you can absolutely apply an access list to allow this traffic. Since you are new to ASA's, I would recommend using the ASDM then for looking at this issue.
Can you post the source-destination IPs and the ACL being implemented that you are using. I should be able to help from there. The other thing you can do though is that when you are seeing the logs scroll by in the ASDM tool you can click on the entry stating the denied packet and from there you should be able to right click or something and have it direct you to the ACL entry that is denying the packet also (I haven't done it yet before though but read it is possible) you are supposed to be able to create a new rule that would allow the denied traffic
ASKER
Can you see it now? I am new in these blogs so I can see it but I think it had to do with the settings.
comm.txt
comm.txt
first, i'm assuming that ACL you posted is applied to the management interface in the inbound direction.
second, do you have a route for the 10.4.2.0/24 network since its wanting to go out the office interface for some reason. It may be failing because it doesn't know how to forward the traffic appropriately.
second, do you have a route for the 10.4.2.0/24 network since its wanting to go out the office interface for some reason. It may be failing because it doesn't know how to forward the traffic appropriately.
ASKER
Hello Cyclops!,
Well The ACL is applied to the office interface, not to the management, I have no access lists applied to the management interface so far.
The office network (10.4.2.0/24) has a connection to the ASA through interfaceGigabitEthernet0/
I do have a route for 10.4.2.0 through interface 3.701.
and I even have a route for the management network (through the management interface GigabitEthernet0/3.54), as well even though it is a direct connection to the ASA.
but still what I cant comprehend is why my office can ping a few devices in the managemente network but not all and still the ASA can reach all of them.
I have been doing packet captures as well and still if I ping I can see only the request, no reply from a few devices from the management network to my office network. But all respond to the ASA.
Do you by any chance know any way to know if the ASA is actually sending the packet (which is not packet capture because it doesnt appear), I am beginning to think that maybe the devices are the ones not responding, or they are not getting the icmp replies for some reason....
Thanks again!!!!
Regards,
Well The ACL is applied to the office interface, not to the management, I have no access lists applied to the management interface so far.
The office network (10.4.2.0/24) has a connection to the ASA through interfaceGigabitEthernet0/
I do have a route for 10.4.2.0 through interface 3.701.
and I even have a route for the management network (through the management interface GigabitEthernet0/3.54), as well even though it is a direct connection to the ASA.
but still what I cant comprehend is why my office can ping a few devices in the managemente network but not all and still the ASA can reach all of them.
I have been doing packet captures as well and still if I ping I can see only the request, no reply from a few devices from the management network to my office network. But all respond to the ASA.
Do you by any chance know any way to know if the ASA is actually sending the packet (which is not packet capture because it doesnt appear), I am beginning to think that maybe the devices are the ones not responding, or they are not getting the icmp replies for some reason....
Thanks again!!!!
Regards,
ok, that's why then.
management - security level 70
office - security level 85
going from lower security level (less trusted) to higher security level (more trusted) is by default denied. You have to create an acl that allows the traffic you want and apply it in the inbound direction on the management interface. That will fix your issue (unless there is something else I'm not aware of but this for sure is blocking it right now).
the asa can reach all of them because there are parts of the packet life cycle that are skipped due to the asa being the source. as to why the office can ping some and not others, I'm not seeing a packet trace for the echo reply from mgmt to office for the host being pinged. since there is no acl on the mgmt interface, my guess is that you might have icmp inspection turned on but would need to see your class-map, policy map, and service policy config. however your packet trace shows starting from a mgmt side host so due to security levels is guaranteed to be firewalled.
you can run a capture actually on the asa. you just create an acl first though otherwise too much to look thru that targets your test. apply it with a capture command, run the test, and then look at the results with show capture.
however first you may want to add an acl to the mgmt int to allow the needed traffic. if not mgmt hosts need to initiate traffic to the office hosts and you're just talking about pinging then please make sure "inspect icmp" is included in the applied service policy. if you post that relevant config I can help you get it added.
management - security level 70
office - security level 85
going from lower security level (less trusted) to higher security level (more trusted) is by default denied. You have to create an acl that allows the traffic you want and apply it in the inbound direction on the management interface. That will fix your issue (unless there is something else I'm not aware of but this for sure is blocking it right now).
the asa can reach all of them because there are parts of the packet life cycle that are skipped due to the asa being the source. as to why the office can ping some and not others, I'm not seeing a packet trace for the echo reply from mgmt to office for the host being pinged. since there is no acl on the mgmt interface, my guess is that you might have icmp inspection turned on but would need to see your class-map, policy map, and service policy config. however your packet trace shows starting from a mgmt side host so due to security levels is guaranteed to be firewalled.
you can run a capture actually on the asa. you just create an acl first though otherwise too much to look thru that targets your test. apply it with a capture command, run the test, and then look at the results with show capture.
however first you may want to add an acl to the mgmt int to allow the needed traffic. if not mgmt hosts need to initiate traffic to the office hosts and you're just talking about pinging then please make sure "inspect icmp" is included in the applied service policy. if you post that relevant config I can help you get it added.
ASKER
Hello Cyclops!!!,
Please find bellow the configuration. A little simple hehe.
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect icmp
inspect ipsec-pass-thru
!
service-policy global_policy global
prompt hostname context
Thank you soo much!!!
Kumo..
Please find bellow the configuration. A little simple hehe.
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect icmp
inspect ipsec-pass-thru
!
service-policy global_policy global
prompt hostname context
Thank you soo much!!!
Kumo..
you already have icmp inspection so return echo replies would be allowed for office side hosts pinging mgmt side hosts.
can you run a packet trace for the host on the office side to the mgmt side that is failing? it should be getting to the device and as long as an echo reply is being sent it should return.
also, while you're at it, run a capture from the asa itself
Capture examples: http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080a9edd6.shtml
can you run a packet trace for the host on the office side to the mgmt side that is failing? it should be getting to the device and as long as an echo reply is being sent it should return.
also, while you're at it, run a capture from the asa itself
Capture examples: http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080a9edd6.shtml
ASKER
Hello Cyclops,
I tried that packet capture before and from the office to the management it works (at least on the packet tracer).
I also did a capture and I still can see only requests, not replys, what I am not sure is if is the ASA the one that is actually blocking, or is the device not responding for some reason.
I previously did wireshark on the ASA port for the management VLAN, but I can see exactly the same as the packet capture, just requests but no replies from the 10.2.1.4, unless the ASA itself pings this host.
Please find attached the file with the info... not really sure if my packet capture is what you asked?
Thanks!!!!
Regards,
comm2.txt
I tried that packet capture before and from the office to the management it works (at least on the packet tracer).
I also did a capture and I still can see only requests, not replys, what I am not sure is if is the ASA the one that is actually blocking, or is the device not responding for some reason.
I previously did wireshark on the ASA port for the management VLAN, but I can see exactly the same as the packet capture, just requests but no replies from the 10.2.1.4, unless the ASA itself pings this host.
Please find attached the file with the info... not really sure if my packet capture is what you asked?
Thanks!!!!
Regards,
comm2.txt
based on that I'm thinking either one of two things.
1) the host being pinged has a firewall that limits what is allowed to ping it.
the ASA just happens to be in that list
2) the destination host isn't configured with an appropriate default gateway.
the ASA would be using the mgmt IP address to ping the host so since it is on the same subnet, no routing (hence no default gateway usage necessary) happens
can you log onto the 10.2.1.4 host and ping an office host. we know that it'll be dropped by the asa, but because there is no echo-reply even being seen by the asa I'm leaning toward this not being an asa problem. the asa looks to be configured just fine. I think its a host misconfiguration issue. have you ran a capture from the mgmt host? what kind of host are you pinging at 10.2.1.4?
1) the host being pinged has a firewall that limits what is allowed to ping it.
the ASA just happens to be in that list
2) the destination host isn't configured with an appropriate default gateway.
the ASA would be using the mgmt IP address to ping the host so since it is on the same subnet, no routing (hence no default gateway usage necessary) happens
can you log onto the 10.2.1.4 host and ping an office host. we know that it'll be dropped by the asa, but because there is no echo-reply even being seen by the asa I'm leaning toward this not being an asa problem. the asa looks to be configured just fine. I think its a host misconfiguration issue. have you ran a capture from the mgmt host? what kind of host are you pinging at 10.2.1.4?
ASKER
Hello Cyclops,
Thanks again :D !!
I can do that tomorrow, the 10.2.1.4 is managed by a vendor it has RIPv2 and is the management network for a few Microwaves and I will need those details so most probably will be posting more details tomorrow.
I will be back!!!
Thanks!!!!
Thanks again :D !!
I can do that tomorrow, the 10.2.1.4 is managed by a vendor it has RIPv2 and is the management network for a few Microwaves and I will need those details so most probably will be posting more details tomorrow.
I will be back!!!
Thanks!!!!
ASKER
Hello Cyclops,
Well something really weird happened after the tests, I did I diagram to see if it helps u to understand. frankly after the test I got even more confused.
On the Diagram both vlans come to one interface on the ASA (as a subinterface for their vlan of course). and the ASA is the gateway.
then IPS 10.2.1.3, 10.2.1.4 and 10.2.1.2 are all coming from the same interface into my switch.
I went to the other extreme, where my managed devices are and did the packet capture with wireshark, I am able to see the requests but not reply, so I thought is the managed device it is not answering for some reason that is the issue. Then in order to prove this I connected my computer to the same interface, pinged the gateway successfullyperfect! , and then when I tried to ping my laptop from the office nothing happened. IT is just the same thing that with the managed device, I can reach the ASA, but cant reach the Office network.
Wireshark shows me all the packets coming to that vlan but is like if the PC doesnt recognize it is for it, I dont send replies, only to the gateway. I can just see the requests coming.
Also, I was able to ping 10.2.1.2 and 10.2.1.3 from the office before, but when I unplugged this connection in order to test with my PC and then put the original cable back into my switch I lost all communication with that network. so basically i can ping from the ASA to these addresses but thats it, nothing more.
This just got me even more confused.
I hope this makes some sense for u, cause it is just killing me at the moment hehe.
Thanks again!!!
Regards,
2013-06-01-14.53.46.jpg
Well something really weird happened after the tests, I did I diagram to see if it helps u to understand. frankly after the test I got even more confused.
On the Diagram both vlans come to one interface on the ASA (as a subinterface for their vlan of course). and the ASA is the gateway.
then IPS 10.2.1.3, 10.2.1.4 and 10.2.1.2 are all coming from the same interface into my switch.
I went to the other extreme, where my managed devices are and did the packet capture with wireshark, I am able to see the requests but not reply, so I thought is the managed device it is not answering for some reason that is the issue. Then in order to prove this I connected my computer to the same interface, pinged the gateway successfullyperfect! , and then when I tried to ping my laptop from the office nothing happened. IT is just the same thing that with the managed device, I can reach the ASA, but cant reach the Office network.
Wireshark shows me all the packets coming to that vlan but is like if the PC doesnt recognize it is for it, I dont send replies, only to the gateway. I can just see the requests coming.
Also, I was able to ping 10.2.1.2 and 10.2.1.3 from the office before, but when I unplugged this connection in order to test with my PC and then put the original cable back into my switch I lost all communication with that network. so basically i can ping from the ASA to these addresses but thats it, nothing more.
This just got me even more confused.
I hope this makes some sense for u, cause it is just killing me at the moment hehe.
Thanks again!!!
Regards,
2013-06-01-14.53.46.jpg
sorry, more questions now.
1) they are different vlans off the same physical interface of the asa but your diagram shows them going into different devices. Can you supply a physical diagram as well as a logical diagram?
2) you say the mgmt network hosts are off the same interface of the switch. can you elaborate how that is so?
3) did you verify the host default gateway configuration? because the host is on the mgmt network it'll be able to ping the asa interface. look at the route table and verify that for the office network it has route back.
4) when you connected your laptop, did you run wireshark from it and at least see a echo reply being sent? I understand the asa might not have seen one but we need to verify that the managed devices are actually sending them or not. so if the laptop is sending on and the asa doesn't see one then something in between in the physical flow is causing it to be dropped.
1) they are different vlans off the same physical interface of the asa but your diagram shows them going into different devices. Can you supply a physical diagram as well as a logical diagram?
2) you say the mgmt network hosts are off the same interface of the switch. can you elaborate how that is so?
3) did you verify the host default gateway configuration? because the host is on the mgmt network it'll be able to ping the asa interface. look at the route table and verify that for the office network it has route back.
4) when you connected your laptop, did you run wireshark from it and at least see a echo reply being sent? I understand the asa might not have seen one but we need to verify that the managed devices are actually sending them or not. so if the laptop is sending on and the asa doesn't see one then something in between in the physical flow is causing it to be dropped.
ASKER
Dont worry cyclops, I really appreciate the help ! feel free to ask.
1. Diagramas are attached.
2. These three devices I have there are the management ports for a wireless link, and each of them has a small 4 port hub included as part of the hardware. so one port is going to my switch, and the others are daisy chained to each other. I hope the diagrams can make more sense for you?
3. Host default gateway was correct, and they can ping the gateway as well. both networks are reached from the ASA through an static route.
4. This was the weirdest of all the tests. I did put wireshark on my laptop and connected straight into the port that goes to my switch, I was able to see all the incoming packets but never saw a reply from my laptop. as if for some reason my laptop wasnt replying.
It had the correct gateway though and it was able to ping it. It will just sort of ignore everything is not coming from the gateway.
If i connect to the managed devices and run wireshark from them is just exactly the same, packets are coming but no reply. I thought it was a failure with the devices, but I know my laptop works fine and couldnt get any reply from it on thw wireshark capture either.
Thanks again!!! I really really appreciate the help!
Regards,
Logical.jpg
physical.jpg
1. Diagramas are attached.
2. These three devices I have there are the management ports for a wireless link, and each of them has a small 4 port hub included as part of the hardware. so one port is going to my switch, and the others are daisy chained to each other. I hope the diagrams can make more sense for you?
3. Host default gateway was correct, and they can ping the gateway as well. both networks are reached from the ASA through an static route.
4. This was the weirdest of all the tests. I did put wireshark on my laptop and connected straight into the port that goes to my switch, I was able to see all the incoming packets but never saw a reply from my laptop. as if for some reason my laptop wasnt replying.
It had the correct gateway though and it was able to ping it. It will just sort of ignore everything is not coming from the gateway.
If i connect to the managed devices and run wireshark from them is just exactly the same, packets are coming but no reply. I thought it was a failure with the devices, but I know my laptop works fine and couldnt get any reply from it on thw wireshark capture either.
Thanks again!!! I really really appreciate the help!
Regards,
Logical.jpg
physical.jpg
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Hello Cyclops,
You were totally right, it was an issue with the managed devices, i had to wait for the vendor to come in and check the devices himself, he had some routing configured within the managed devices which was causing an issue. Even though he assured there was a gateway and no routing, when he came and we both entered to the devices we found the following:
1. Rip was configured in a wrong way,
2. there was no gateway in any of them.
3. routing tables were sending traffic somewhere else
Therefore i assume that the old switches with the old EXtreme IOS 12.2.x had a bug that somehow made that setup to work all these years, and when we introduced the new EXOS 15.x.x it actually worked as it was supposed to?. Or maybe with the changes the managed devices got blocked ??? I have no idea, but i really appreciate your help !!!
Thanks a lot !!!
You were totally right, it was an issue with the managed devices, i had to wait for the vendor to come in and check the devices himself, he had some routing configured within the managed devices which was causing an issue. Even though he assured there was a gateway and no routing, when he came and we both entered to the devices we found the following:
1. Rip was configured in a wrong way,
2. there was no gateway in any of them.
3. routing tables were sending traffic somewhere else
Therefore i assume that the old switches with the old EXtreme IOS 12.2.x had a bug that somehow made that setup to work all these years, and when we introduced the new EXOS 15.x.x it actually worked as it was supposed to?. Or maybe with the changes the managed devices got blocked ??? I have no idea, but i really appreciate your help !!!
Thanks a lot !!!
clear access-list acl_name_here counters
then run your test and look at the counters
sh access-list acl_name_here
chances are that a >0 count will be seen on more than just the entry that is affecting your traffic but it should hopefully at least narrow it down.