admitech
asked on
Cannot reach website inside our domain with 0 restrictions, but can from outside our network. Please help!
Site: https://sftp.bpas.com/
Accessible outside our network not inside
Equipment: Cisco ASA. No restrictions outbound
Two outbound connection possibilities: CNYwireless, Time Warner. Doesn't work via either.
From inside our network I get:
"This site can’t be reached
The webpage at https://sftp.bpas.com/ might be temporarily down or it may have moved permanently to a new web address."
DNS is working fine. Nslookup fine.
Tracert hits our gateway, and outside fine and then dies (as if it doesn't know how to get back...). All other sites are fine.
Not a cert issue or individual or Browser issue....
Accessible outside our network not inside
Equipment: Cisco ASA. No restrictions outbound
Two outbound connection possibilities: CNYwireless, Time Warner. Doesn't work via either.
From inside our network I get:
"This site can’t be reached
The webpage at https://sftp.bpas.com/ might be temporarily down or it may have moved permanently to a new web address."
DNS is working fine. Nslookup fine.
Tracert hits our gateway, and outside fine and then dies (as if it doesn't know how to get back...). All other sites are fine.
Not a cert issue or individual or Browser issue....
You need DNS doctoring.
At the end of the static nat statement for that server, add the keyword "dns".
At the end of the static nat statement for that server, add the keyword "dns".
For doctoring to work, you'll also need to route all DNS queries and replies through the ASA so they can be translated. Additionally, DNS inspection must be turned on on the ASA. You could also implement u-turning which tells the ASA to doctor both the source and destination IP.
More detail here: https://supportforums.cisco.com/document/145401/dns-doctoring-and-u-turning-asa-when-and-how-use-it
More detail here: https://supportforums.cisco.com/document/145401/dns-doctoring-and-u-turning-asa-when-and-how-use-it
This only is a concern when internal DNS is configured to hand out a public IP instead of the internal.
I've yet to see that happen.
I've yet to see that happen.
ASKER
No this never worked. Stepped on this landmine. DNS seems to be fine from all devices including the ASA so would DNS inspection/u-turning assist with this still? It seems to get out but not return.
What is your DNS server while inside the network?
Is it external?
if yes, add the "dns" keyword to the end of the static statement and you're done
if no, then it's internal
does your internal server forward queries outside for the domain?
if yes, then you're done after you add the "dns" keyword
if no, does your server give you a public IP?
if no, you're done (and you should be able to reach it)
if yes, change it to the inside IP
Is it external?
if yes, add the "dns" keyword to the end of the static statement and you're done
if no, then it's internal
does your internal server forward queries outside for the domain?
if yes, then you're done after you add the "dns" keyword
if no, does your server give you a public IP?
if no, you're done (and you should be able to reach it)
if yes, change it to the inside IP
@Jan
Interesting. I was under the impression that you needed to pass all your DNS queries through the ASA for doctoring to work regardless of where DNS lookups were taking place. I learned something today. Thanks, Jan!
@admitech
Yes, either method should fix your issue.
Interesting. I was under the impression that you needed to pass all your DNS queries through the ASA for doctoring to work regardless of where DNS lookups were taking place. I learned something today. Thanks, Jan!
@admitech
Yes, either method should fix your issue.
ASKER
Wouldn't let me edit the last comment. I mentioned U-turning to our Sr. Net Admin and he looked at me like I had 3 heads....
Jan: let me run through that list. Great if this then this...lol! Thanks!
Jan: let me run through that list. Great if this then this...lol! Thanks!
Ha! You should let your senior admin fix it then. Be sure to save a copy of your config first!
Or hire Jan. She's available for gigs.
Well, the flow chart says that if the packets hit the firewall to go to an outside address and that address is a static NAT with "dns", the ASA should swap the IP in the packets and send it back.
ASKER
Lol, unfortunately I arrived at "if no, you're done (and you should be able to reach it)" and cannot!
ASKER
Jan/Gilnov:
Want to paste the config. You guys can probably figure it out in two seconds?!
Want to paste the config. You guys can probably figure it out in two seconds?!
sh run nat
copy and paste the nat entry for the web server into notepad
copy and paste it again
on the first line, you will insert "no" at the beginning.
on the second line, you will add "dns" to the end.
copy both lines and paste in configuration mode in to the ASA (no quotes). save it.
to remove a static entry that is in use will require that you clear the translation first. my recommendation is to do this during a non-peak period -- like before the office opens or whenever traffic to that server is low.
copy and paste the nat entry for the web server into notepad
copy and paste it again
on the first line, you will insert "no" at the beginning.
on the second line, you will add "dns" to the end.
copy both lines and paste in configuration mode in to the ASA (no quotes). save it.
to remove a static entry that is in use will require that you clear the translation first. my recommendation is to do this during a non-peak period -- like before the office opens or whenever traffic to that server is low.
send it via EE message, please don't post it here.
If that server is hosted internally (and it sounds like it is), why not use a split DNS configuration to resolve sftp.bpas.com to the server's internal address? Then you don't have to modify the router/firewall config at all, as that traffic will stay inside your network.
The simplest way to do this will be to create a forward lookup zone named sftp.bpas.com in your internal DNS, then create a blank host record in that zone. Give that record the server's internal IP address. Flush the resolver cache on a client (ipconfig /flushdns) and test from that client.
The simplest way to do this will be to create a forward lookup zone named sftp.bpas.com in your internal DNS, then create a blank host record in that zone. Give that record the server's internal IP address. Flush the resolver cache on a client (ipconfig /flushdns) and test from that client.
ASKER
its in an external webpage outside of our organization and not associated with us. Sorry if I confused!
We'll, that does shed a different light. Can we assume all PC's inside your LAN have the same problem (or at least all PC's you checked)? What about your domain controller (assuming it's also your internal DNS server)? Where are your PC's getting their EXTERNAL DNS queries served (e.g. do they just check with the DC and the DC handles external lookups or do all machines go directly to an external DNS provider, including the DC/DNS server)?
Also, you said tracert dies but where does the trace stop? Can you tell if it ever leaves your carrier's network?
Also, you said tracert dies but where does the trace stop? Can you tell if it ever leaves your carrier's network?
It's does not appear to be with the ASA and it's not DNS.
ASKER
Gilnov:
Yes you are right. None of the PCs on the network can get there. They check with the DC/DNS. DNS appears fine all around. Tracert dies after it leaves the carrier's network it seems.....
Yes you are right. None of the PCs on the network can get there. They check with the DC/DNS. DNS appears fine all around. Tracert dies after it leaves the carrier's network it seems.....
I haven't see your config. Is port 22 open on both the internal and external interfaces? Does the remote site use different/non-standard ports for SSH/SFTP? If so, are those ports open on your ASA?
Also, can you confirm for certain that people inside can NOT access the site AT THE SAME TIME that people outside your LAN can access the site? That is to say, have you ruled out the possibility that the site might actually be temporarily unavailable?
It's not an ASA issue. This is an issue reaching an https _remote_ site.
@Jan
It would seem to be something on the OP's network though since the site is accessible from other networks, wouldn't you agree?
It would seem to be something on the OP's network though since the site is accessible from other networks, wouldn't you agree?
I cannot rule that out yet.
ASKER
Yep as Jan said this is not a site to site. The site we are trying to reach is an investment site that HR from a lot of companies do business with but not apart of our network, etc. It seems to be an HTTPs issue but cannot figure out why. Last place it dies here (on a tracert)
106 ms 90 ms 27 ms ae-1-4.bar2.buffalo1.level
106 ms 90 ms 27 ms ae-1-4.bar2.buffalo1.level
The purpose of the traceroute is to see if you made it off-net to rule out an internal problem. You do and that the traceroute dies may only mean that it is blocked in the path.
ASKER
Yes. True, just wanted to show last hop is outside of us and seemingly our carrier....Nevermind that I included that lol.
@admitech
So all we know at this point is your packets are getting out but responses are not coming back from that site.
Are other ssh/sftp sites affected?
Have you tried connecting from inside your network with PuTTY SFTP or similar client (i.e. a client other than a web browser)?
So all we know at this point is your packets are getting out but responses are not coming back from that site.
Are other ssh/sftp sites affected?
Have you tried connecting from inside your network with PuTTY SFTP or similar client (i.e. a client other than a web browser)?
ASKER
@gilnov
Exactly.
No other sites are affected but this ONE. That we know of; apparently this is a year long issue that I stepped on but not giving up until this is fixed.
@Jan @gilnov
I tried Putty with Jan. Jan what was your conclusion on this other than obviously we proved:
Not the ASA, not DNS?
By the way guys I totally appreciate you sticking with me on this! I've got a 10am meeting with the "big dogs" so well see what suggestions we get BUT considering this has been known not working for a year I am not expecting much! And of course now its urgent! You know how that works! Good ole IT!
Exactly.
No other sites are affected but this ONE. That we know of; apparently this is a year long issue that I stepped on but not giving up until this is fixed.
@Jan @gilnov
I tried Putty with Jan. Jan what was your conclusion on this other than obviously we proved:
Not the ASA, not DNS?
By the way guys I totally appreciate you sticking with me on this! I've got a 10am meeting with the "big dogs" so well see what suggestions we get BUT considering this has been known not working for a year I am not expecting much! And of course now its urgent! You know how that works! Good ole IT!
DNS works and he getting the correct IP address.
Packet-tracer on the ASA shows the flows permitted.
Traceroute shows the packets making it several hops -- way off net before they time out.
There has to be a filter in routing toward the destination or at the destination server that needs to be updated is my best guess.
Packet-tracer on the ASA shows the flows permitted.
Traceroute shows the packets making it several hops -- way off net before they time out.
There has to be a filter in routing toward the destination or at the destination server that needs to be updated is my best guess.
Have you contacted your ISP(s) to see if they can offer any insight into why the route is being blocked?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I was thinking maybe the ISP could peer deeper into the connection attempt from their perspective and shed additional light on what happens when it leaves their edge where it times out.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
True enough but they might be able to see the reason the connection is timing out. Probably not but it can't hurt to ask. We're still trying to find the edges of the problem.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
@Gilnov and @Jan
I love you both. Seriously owe you guys whatever you drink. A lot of work to find out that after contacting bpas for the 20th time I found the right guy who says "OH, HEY! you are blocked! Too many logon failures! They removed my ip and "TA DA"!!! It works!
I'm going to go out on a limb and say the Time Warner IP is on the block list too!
I love you both. Seriously owe you guys whatever you drink. A lot of work to find out that after contacting bpas for the 20th time I found the right guy who says "OH, HEY! you are blocked! Too many logon failures! They removed my ip and "TA DA"!!! It works!
I'm going to go out on a limb and say the Time Warner IP is on the block list too!
ASKER
Thanks so much for going ABOVE AND WAY BEYOND. Jan you better keep in touch I owe you for your efforts!
Happy to be of assistance. Write down the person or at least the department you spoke with at BPAS. If you were blocked for too many failed logons once, it's very likely you'll end up being blocked again.
Did it ever work? If so, when did it stop and what changed between now and the last time it worked? Is there a piece of gear turned off, unplugged or dead?