Tropical_Computer
asked on
PC in remote site can't access shares on domain controller
I have been having ongoing issues with computers trying to access shares on a domain controller from a remote location. I think my issue may be AD configuration, though I am not ruling out issues with the gateway-to-gateway VPN between two Netgear UTM9S routers. The two networks are on different subnets.
I have a domain controller (10.100.1.10) running Server 2003 R2 and a few file shares with permissions set to a few AD groups (less than 10). PC's at the remote site were joined to the domain on the HQ network before being moved to the remote site (10.100.2.61).
The remote PC can browse file shares and access ones like NETLOGON and SYSVOL on the DC. Shares with permissions set to everyone can be opened, however downloading files never succeeds, appearing to time out. The remote PC is using DNS from the domain controller that appears to be working fine.
I ran PortQryUI from the computer and found that a number of ports on the DC are blocked and I'm not sure why, specifically:
TCP port 389 (ldap service): LISTENING
Using ephemeral source port
Sending LDAP query to TCP port 389...
LDAP query to port 389 failed
Server did not respond to LDAP query
UDP port 389 (unknown service): LISTENING or FILTERED
Using ephemeral source port
Sending LDAP query to UDP port 389...
LDAP query to port 389 failed
Server did not respond to LDAP query
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
Failed to resolve IP address to name
querying...
TCP port 3269 (msft-gc-ssl service): FILTERED
portqry.exe -n 10.100.1.10 -e 3269 -p TCP exits with return code 0x00000002
Windows Firewall on the DC is turned off and there are no active rules on the routers that should be blocking them.
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
Failed to resolve IP address to name
querying...
TCP port 53 (domain service): FILTERED
UDP port 53 (domain service): LISTENING or FILTERED
Sending DNS query to UDP port 53...
DNS query timed out
portqry.exe -n 10.100.1.10 -e 53 -p BOTH exits with return code 0x00000002.
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
IP address resolved to CEMSSV-01
querying...
TCP port 88 (kerberos service): LISTENING
UDP port 88 (kerberos service): LISTENING or FILTERED
portqry.exe -n 10.100.1.10 -e 88 -p BOTH exits with return code 0x00000002.
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
IP address resolved to CEMSSV-01
querying...
UDP port 138 (netbios-dgm service): LISTENING or FILTERED
portqry.exe -n 10.100.1.10 -e 138 -p UDP exits with return code 0x00000002.
~~
In DNS, I do have a reverse lookup zone for the other subnet but have taken no other configuration action. And in Active Directory Sites and Services, I did create a separate site and subnet. So I thought I have everything set up properly and to be clear, everything works just fine in the main network.
This leads me to think that either a) something on the server is blocking the traffic even though the firewall is disabled, b) some permission needs to be set on the DC to allow these requests to succeed on the other subnet, or c) something is otherwise not configured in DNS and/or AD for this remote site. the very slow performance between the sites has me wondering why, as its a 15/5 connection on either side with low utilization.
Thanks for the help! Please tell me I missed something, that would make my day!
I have a domain controller (10.100.1.10) running Server 2003 R2 and a few file shares with permissions set to a few AD groups (less than 10). PC's at the remote site were joined to the domain on the HQ network before being moved to the remote site (10.100.2.61).
The remote PC can browse file shares and access ones like NETLOGON and SYSVOL on the DC. Shares with permissions set to everyone can be opened, however downloading files never succeeds, appearing to time out. The remote PC is using DNS from the domain controller that appears to be working fine.
I ran PortQryUI from the computer and found that a number of ports on the DC are blocked and I'm not sure why, specifically:
TCP port 389 (ldap service): LISTENING
Using ephemeral source port
Sending LDAP query to TCP port 389...
LDAP query to port 389 failed
Server did not respond to LDAP query
UDP port 389 (unknown service): LISTENING or FILTERED
Using ephemeral source port
Sending LDAP query to UDP port 389...
LDAP query to port 389 failed
Server did not respond to LDAP query
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
Failed to resolve IP address to name
querying...
TCP port 3269 (msft-gc-ssl service): FILTERED
portqry.exe -n 10.100.1.10 -e 3269 -p TCP exits with return code 0x00000002
Windows Firewall on the DC is turned off and there are no active rules on the routers that should be blocking them.
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
Failed to resolve IP address to name
querying...
TCP port 53 (domain service): FILTERED
UDP port 53 (domain service): LISTENING or FILTERED
Sending DNS query to UDP port 53...
DNS query timed out
portqry.exe -n 10.100.1.10 -e 53 -p BOTH exits with return code 0x00000002.
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
IP address resolved to CEMSSV-01
querying...
TCP port 88 (kerberos service): LISTENING
UDP port 88 (kerberos service): LISTENING or FILTERED
portqry.exe -n 10.100.1.10 -e 88 -p BOTH exits with return code 0x00000002.
Querying target system called:
10.100.1.10
Attempting to resolve IP address to a name...
IP address resolved to CEMSSV-01
querying...
UDP port 138 (netbios-dgm service): LISTENING or FILTERED
portqry.exe -n 10.100.1.10 -e 138 -p UDP exits with return code 0x00000002.
~~
In DNS, I do have a reverse lookup zone for the other subnet but have taken no other configuration action. And in Active Directory Sites and Services, I did create a separate site and subnet. So I thought I have everything set up properly and to be clear, everything works just fine in the main network.
This leads me to think that either a) something on the server is blocking the traffic even though the firewall is disabled, b) some permission needs to be set on the DC to allow these requests to succeed on the other subnet, or c) something is otherwise not configured in DNS and/or AD for this remote site. the very slow performance between the sites has me wondering why, as its a 15/5 connection on either side with low utilization.
Thanks for the help! Please tell me I missed something, that would make my day!
ASKER
Ping response ranged from 12ms to 28ms and there were timeouts.
Traceroutes going both ways look okay. I'm assuming hop 2 on both sides is the ISP (Cox).
The workstation is part of the domain, having previously lived on the main network. I am using my domain admin account for testing in both sites and can confirm it isn't locked.
Its also worth mentioning that group policy updates are not making their way to the client so I would assume password changes would also fail.
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 95, Lost = 5 (5% loss),
Approximate round trip times in milli-seconds:
Minimum = 14ms, Maximum = 35ms, Average = 18ms
Traceroutes going both ways look okay. I'm assuming hop 2 on both sides is the ISP (Cox).
C:\Users\itadmin.EMR>tracert cemssv-01
Tracing route to cemssv-01 [10.100.1.10]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 10.100.2.1
2 * * * Request timed out.
3 17 ms 15 ms 16 ms CEMSSV-01 [10.100.1.10]
Trace complete.
C:\Documents and Settings\ITAdmin>tracert emrpc-05
Tracing route to emrpc-05 [10.100.2.101]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 10.100.1.1
2 * * * Request timed out.
3 17 ms 15 ms 19 ms EMRPC-05 [10.100.2.101]
Trace complete.
C:\Documents and Settings\ITAdmin>
The workstation is part of the domain, having previously lived on the main network. I am using my domain admin account for testing in both sites and can confirm it isn't locked.
Its also worth mentioning that group policy updates are not making their way to the client so I would assume password changes would also fail.
Log Name: System
Source: Microsoft-Windows-GroupPolicy
Date: 2/2/2013 7:11:21 PM
Event ID: 1006
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: EMRPC-05.emr.local
Description:
The processing of Group Policy failed. Windows could not authenticate to the Active Directory service on a domain controller. (LDAP Bind function call failed). Look in the details tab for error code and description.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-GroupPolicy" Guid="{AEA1B4FA-97D1-45F2- A64C-4D69F FFD92C9}" />
<EventID>1006</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>1</Opcode>
<Keywords>0x8000000000000000</Keywor ds>
<TimeCreated SystemTime="2013-02-03T00:11:21.7252 75300Z" />
<EventRecordID>48439</EventRecordID>
<Correlation ActivityID="{B982FF17-46DA-415A-B3A0 -69A6EC04D 5A2}" />
<Execution ProcessID="1012" ThreadID="3988" />
<Channel>System</Channel>
<Computer>EMRPC-05.emr.local</Comput er>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="SupportInfo1">1</Data>
<Data Name="SupportInfo2">5012</Data>
<Data Name="ProcessingMode">0</Data>
<Data Name="ProcessingTimeInMilliseconds"> 241459</Da ta>
<Data Name="ErrorCode">81</Data>
<Data Name="ErrorDescription">Server Down</Data>
<Data Name="DCName">
</Data>
</EventData>
</Event>
Looks like there's some trouble accessing the LDAP and/or domain resources.
I'm sorry, but I'm not that familiar with PortQryUI.
Can you run an nmap of the server from the workstation?
http://nmap.org
Nmap -v -A IP_Address
Also, you forgot the nslookup part from my last comment.
Start -> Run -> cmd
nslookup
(and then just type in some server and workstation names you know of and see if it responds back with an answer.)
I'm sorry, but I'm not that familiar with PortQryUI.
Can you run an nmap of the server from the workstation?
http://nmap.org
Nmap -v -A IP_Address
Also, you forgot the nslookup part from my last comment.
Start -> Run -> cmd
nslookup
(and then just type in some server and workstation names you know of and see if it responds back with an answer.)
ASKER
Sorry, forgot to post the nslookups
From the workstation...
From the server...
The nmap results going to and from the server are attached.
nmap--server-to-workstation.txt
nmap--workstation-to-server.txt
From the workstation...
C:\Users\itadmin.EMR>nslookup cemssv-01
Server: cemssv-01.emr.local
Address: 10.100.1.10
Name: cemssv-01.emr.local
Address: 10.100.1.10
C:\Users\itadmin.EMR>nslookup emrpc-01
Server: cemssv-01.emr.local
Address: 10.100.1.10
Name: emrpc-01.emr.local
Address: 10.100.1.101
From the server...
C:\Documents and Settings\ITAdmin>nslookup emrpc-05
Server: localhost
Address: 127.0.0.1
Name: emrpc-05.emr.local
Address: 10.100.2.101
C:\Documents and Settings\ITAdmin>nslookup emrpc-01
Server: localhost
Address: 127.0.0.1
Name: emrpc-01.emr.local
Address: 10.100.1.101
The nmap results going to and from the server are attached.
nmap--server-to-workstation.txt
nmap--workstation-to-server.txt
ASKER
I just tried to access the server shares from the workstation and it failed again. I re-ran nmap from the workstation to the server and did a compare, there was 1 difference.
Where previously
I am not running any kind of mail services on the server so I'm not sure this matters in the global scheme of things.
Thanks for looking at this for me!
25/tcp open smtp?
|_smtp-commands: Couldn't establish connection on port 25
Where previously
Discovered open port 25/tcp on 10.100.2.101
I am not running any kind of mail services on the server so I'm not sure this matters in the global scheme of things.
Thanks for looking at this for me!
The nmap results are from 10.100.2.101 and 2.121
Please provide nmap testing from the 10.100.2.x network to the server. Thanks.
Please provide nmap testing from the 10.100.2.x network to the server. Thanks.
ASKER
Wow, that got all kinds of messed up. Lets try that again.
CEMSSV-01-to-EMRPC-05.txt
EMRPC-05-to-CEMSSV-01.txt
CEMSSV-01-to-EMRPC-05.txt
EMRPC-05-to-CEMSSV-01.txt
I'm sorry man, this is a good one. The ports are available and services are running
I'm going to chalk this one up to VPN issues. 5% doesn't seem like a lot of loss, but try running bigger packets and see what you get.
Ping 10.100.1.10 -n 100 -l 1200
Then try
Ping 10.100.110 -n 100 -l 1600
I'm going to chalk this one up to VPN issues. 5% doesn't seem like a lot of loss, but try running bigger packets and see what you get.
Ping 10.100.1.10 -n 100 -l 1200
Then try
Ping 10.100.110 -n 100 -l 1600
ASKER
Thanks, yeah I know. I'm wondering if there is anything the ISP can do (Cox)? I hate to give their tech support a call as they are often useless for residential things. The overall slowness of the VPN has me wondering too.
But I also can't rule out something in AD that may not be set right. It doesn't make sense to me that I can browse a share, can view the files in a share set to everyone but can't download them, and can't even browse files in shares with permissions set to Domain Users.
A third possibility is there is a hardware problem on one of the Gateways. I've had a ticket open with Netgear for over a month now and they are claiming their stuff is working fine, though.
But I also can't rule out something in AD that may not be set right. It doesn't make sense to me that I can browse a share, can view the files in a share set to everyone but can't download them, and can't even browse files in shares with permissions set to Domain Users.
A third possibility is there is a hardware problem on one of the Gateways. I've had a ticket open with Netgear for over a month now and they are claiming their stuff is working fine, though.
Run those pings and post the results. Could be an MTU issue causing data loss when you send/receive packets over 1500bytes.
ASKER
Well here is something, pinging the server from the workstation...
and
C:\Users\itadmin.EMR>ping 10.100.1.10 -n 100 -l 1200
Pinging 10.100.1.10 with 1200 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
.
.
.
Request timed out.
Request timed out.
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 0, Lost = 100 (100% loss),
and
C:\Users\itadmin.EMR>ping 10.100.1.10 -n 100 -l 1600
Pinging 10.100.1.10 with 1600 bytes of data:
Request timed out.
Request timed out.
.
.
.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 0, Lost = 100 (100% loss),
That's nasty. It's your VPN for sure.
Ok, let's drop the packet size to find out what the cut off is.
Try these two.
Ping 10.100.1.10 -n 100 -l 800
Ping 10.100.1.10 -n 100 -l 400
Talk to Netgear again and show them the results above, that should show them it's there equipment. Better yet, go Cisco, for routing and VPN I don't like anything else. I've heard Juniper is very good too, but I don't have any experience with it.
I've not used Sonicwalls in about 5 years, but back then their VPN was horrible. Never trusted them since.
Anyways, just saying in case you want to switch to something else. First step would probably be to try to fix the VPN tunnel on the Netgears.
Ok, let's drop the packet size to find out what the cut off is.
Try these two.
Ping 10.100.1.10 -n 100 -l 800
Ping 10.100.1.10 -n 100 -l 400
Talk to Netgear again and show them the results above, that should show them it's there equipment. Better yet, go Cisco, for routing and VPN I don't like anything else. I've heard Juniper is very good too, but I don't have any experience with it.
I've not used Sonicwalls in about 5 years, but back then their VPN was horrible. Never trusted them since.
Anyways, just saying in case you want to switch to something else. First step would probably be to try to fix the VPN tunnel on the Netgears.
ASKER
I finally feel a glimmer of hope! So packet size 400 worked, 800 didn't (see below). So I started trying different packet sizes and it looks like 512 is the max packet size that comes through.
I'm already considering other vendors considering the poor support I've received from Netgear. In a month they never asked the questions you are asking. You'd think a company that specializes in networking products would look beyond a basic ping.
Seeing where this is going, I'm assuming I'll need to change the MTU on various devices. Just the firewalls or on the PC's as well? And may Cox (the ISP) need to look at their MTU settings for the account?
Ironically I just found this article on Netgear's website about MTU and VPNs.
http://kb.netgear.com/app/answers/detail/a_id/1153/~/mtu,-partial-loss-of-internet-connection,-and-performance
I'm already considering other vendors considering the poor support I've received from Netgear. In a month they never asked the questions you are asking. You'd think a company that specializes in networking products would look beyond a basic ping.
Seeing where this is going, I'm assuming I'll need to change the MTU on various devices. Just the firewalls or on the PC's as well? And may Cox (the ISP) need to look at their MTU settings for the account?
Ironically I just found this article on Netgear's website about MTU and VPNs.
http://kb.netgear.com/app/answers/detail/a_id/1153/~/mtu,-partial-loss-of-internet-connection,-and-performance
C:\Users\itadmin.EMR>Ping 10.100.1.10 -n 100 -l 800
Pinging 10.100.1.10 with 800 bytes of data:
Request timed out.
Request timed out.
.
.
Request timed out.
Request timed out.
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 0, Lost = 100 (100% loss),
C:\Users\itadmin.EMR>Ping 10.100.1.10 -n 100 -l 400
Pinging 10.100.1.10 with 400 bytes of data:
Reply from 10.100.1.10: bytes=400 time=21ms TTL=62
Reply from 10.100.1.10: bytes=400 time=17ms TTL=62
.
.
Reply from 10.100.1.10: bytes=400 time=18ms TTL=62
Reply from 10.100.1.10: bytes=400 time=19ms TTL=62
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 100, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 16ms, Maximum = 34ms, Average = 20ms
C:\Users\itadmin.EMR>Ping 10.100.1.10 -n 100 -l 512
Pinging 10.100.1.10 with 512 bytes of data:
Reply from 10.100.1.10: bytes=512 time=18ms TTL=62
Reply from 10.100.1.10: bytes=512 time=18ms TTL=62
.
.
Reply from 10.100.1.10: bytes=512 time=19ms TTL=62
Reply from 10.100.1.10: bytes=512 time=18ms TTL=62
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 100, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 16ms, Maximum = 30ms, Average = 19ms
C:\Users\itadmin.EMR>Ping 10.100.1.10 -n 100 -l 513
Pinging 10.100.1.10 with 513 bytes of data:
Request timed out.
Request timed out.
.
.
Request timed out.
Request timed out.
Ping statistics for 10.100.1.10:
Packets: Sent = 100, Received = 0, Lost = 100 (100% loss),
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
That was it! I did have to set the MTU to 512 on both the server and the remote workstation, still assessing whether I need to make this change on the network local to the server.
So, obviously this is a new area to me. Can someone give me the 10,000 foot view of what just happened and why it worked, as well as a pointer to additional reading/learning? And is there a downside to setting the MTU to this value? I'm used to seeing MTU of 1500 as "good".
I'd also be open to recommendations for Cisco SMB routers as that is a tech stack I want to start learning.
So, obviously this is a new area to me. Can someone give me the 10,000 foot view of what just happened and why it worked, as well as a pointer to additional reading/learning? And is there a downside to setting the MTU to this value? I'm used to seeing MTU of 1500 as "good".
I'd also be open to recommendations for Cisco SMB routers as that is a tech stack I want to start learning.
You only need to set the MTU on the local resources used by the remote systems. So unless your remote workstations access files from the local workstations then you don't need to set it on them.
Here's the thing, local Ethernet MTU is set to 1500bytes. That's the standard packet size on an Ethernet packet. When you configure VPN it uses part of those 1500bytes for encryption. So in turn it reduces the actual MTU to somewhere around 1400bytes... Usually.
With that drop in MTU, VPN devices usually take care of fragmenting the packets (one way or another.)
In your case, for whatever reason, the MTU dropped all the way down to 512bytes and your devices weren't fragmenting, or at least not down to that level.
Sorry for being so broad, but it could be lots of reasons why this happened for you.
As for recommendations. Check out the Cisco 1800/1900 lines. Don't go the SMB route other than UC560 which does VoIP. My personal opinion is that the Linksys SMB stuff isn't any better than the Netgear or Dlink stuff.
If its a small office and all you really need is VPN and firewall, check out the Cisco ASA 5505.
Here's the thing, local Ethernet MTU is set to 1500bytes. That's the standard packet size on an Ethernet packet. When you configure VPN it uses part of those 1500bytes for encryption. So in turn it reduces the actual MTU to somewhere around 1400bytes... Usually.
With that drop in MTU, VPN devices usually take care of fragmenting the packets (one way or another.)
In your case, for whatever reason, the MTU dropped all the way down to 512bytes and your devices weren't fragmenting, or at least not down to that level.
Sorry for being so broad, but it could be lots of reasons why this happened for you.
As for recommendations. Check out the Cisco 1800/1900 lines. Don't go the SMB route other than UC560 which does VoIP. My personal opinion is that the Linksys SMB stuff isn't any better than the Netgear or Dlink stuff.
If its a small office and all you really need is VPN and firewall, check out the Cisco ASA 5505.
ASKER
Thought I would share the final resolution of the MTU issue. I can't thank you folks enough for teaching me to troubleshoot via packet size that led to this being fixed!
The root cause wasn't the UTM routers or the ISP at all. It was a Netgear Prosafe JGS524E switch. The installed firmware (1.0.0.9) had a "feature" to help prevent DOS attacks that limited packet sizes to 512. The latest firmware, 1.0.0.19, had this feature removed due to all the complications it caused for other customers.
So once the firmware on the switch was fixed I was able to ping everything at 1500 without issue.
How about them taters?!
Thanks again!
The root cause wasn't the UTM routers or the ISP at all. It was a Netgear Prosafe JGS524E switch. The installed firmware (1.0.0.9) had a "feature" to help prevent DOS attacks that limited packet sizes to 512. The latest firmware, 1.0.0.19, had this feature removed due to all the complications it caused for other customers.
So once the firmware on the switch was fixed I was able to ping everything at 1500 without issue.
How about them taters?!
Thanks again!
Crazy! Thanks for the follow up. This one is definetly going on my knowledgebase for future reference.
Run a traceroute to the server, and another from the server to the workstation. This will make sure you got your routing good.
Run nslookup and test looking up a few workstations and servers on the domain. This will test to make sure your server is responding to dns queries.
Finally, just make sure your workstation is part of the domain. Make sure the computer account is not locked out and that it's authenticating to the domain properly. Change the password on AD and test to make sure the workstation can login with that new password.
Report back and let us know how it goes.