Slow Logoff Times - Remote Desktop 2008R2

Hey,

I've been troubleshooting this issue for a couple days now and haven't been able to get to the bottom of what is going on.
A client of mine has reported that their users have been having really slow log off times for quite a while now (we could be talking years).

- When a user logs off, it hangs at 'Please wait for the User Profile Service...'
- They have a Windows Domain with 3 2008 R2 Remote Desktop Servers setup with load balancing (all VM's).
- They have roaming profiles configured (via AD not GPO) and the average profile is around 300MB.
- The write speeds between the servers is around 50-60MB/sec, so really shouldn't take long to write the profiles back.
- IPv6 is enabled on all servers (except 1 of the 2 DC's).
- DNS has been checked over, all appears correct.
- DCDIAG has been run on both DC's and passes successfully.
- I've tested removing the roaming profile path in AD and it logs off straight away.

Event Viewer has the following warnings:

6005/Winlogon
The winlogon notification subscriber <Profiles> is taking long time to handle the notification event (Logoff).

6006/Winlogon
The winlogon notification subscriber <Profiles> took 103 second(s) to handle the notification event (Logoff).

There is another error across all 3 servers that I don't think is relevant to the issue, but here it is incase someone knows something I don't:

3100/Search
The winlogon notification subscriber <Profiles> took 103 second(s) to handle the notification event (Logoff).

I've thrown a days worth of investigating/googling into this so far and haven't made any progress; any help and advice would be greatly appreciated!
LVL 2
kibatsuAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

yo_beeDirector of Information TechnologyCommented:
kibatsuAuthor Commented:
Yeah, I've read through that one and double checked; all the servers (2 DC's + 3 TS') only have 1 NIC.
The 7 Worst Nightmares of a Sysadmin

Fear not! To defend your business’ IT systems we’re going to shine a light on the seven most sinister terrors that haunt sysadmins. That way you can be sure there’s nothing in your stack waiting to go bump in the night.

kibatsuAuthor Commented:
Hmm, I'll give it a go. I haven't used this tool before...
yo_beeDirector of Information TechnologyCommented:
either have I
kibatsuAuthor Commented:
Any other ideas? I didn't even get to using this tool, it requires a 6GB download.
kibatsuAuthor Commented:
I just tried using Group Policy to configure roaming profiles instead of AD settings and found the exact same delay (2mins 9secs for a blank profile).
No luck there unfortunately.
yo_beeDirector of Information TechnologyCommented:
Do you have verbose GP logging enabled?
This will write to the application log under Group Policy. This might uncover something else.
yo_beeDirector of Information TechnologyCommented:
Just found this hotfix that is meant to address the issue you are experiencing

https://support.microsoft.com/en-us/kb/981830
kibatsuAuthor Commented:
We tried the hotfix, but received the error: "This update is not applicable for your computer"
joharderCommented:
As a test, set the roaming profile path based on IP address instead of server name.  Also, what is the response time to the roaming profile repository from the Remote Desktop server?

Is there QoS in place or any other network configuration that could be causing packet loss?

Install Network Monitor and try running a network trace.
kibatsuAuthor Commented:
Thanks for the reply.
We've tested using an IP instead and it didn't make any difference. We also don't have any QoS set up either.

I'm not familiar with Network Monitor - I did install and run it, but am not sure how to decipher the information from it.
David Johnson, CD, MVPOwnerCommented:
you could use procmon from sysinternals
howto: http://www.howtogeek.com/school/sysinternals-pro/lesson5/all/
Download and iinstructions
or even better use the windows performance toolkit and run it on one machine (you need to get the appropriate tool for the client O/S
Lionel MMSmall Business IT ConsultantCommented:
Is this happening to all users? can you create a new user, let's say test1 w/o roaming profile and have it logon and and off and see what it does. Then give it a roaming profile and notice the differences if any--keep in the mind the first time will be quite long as it copies the roaming profile to the server. also check the DNS setting on the users logging on; their local DNS settings.
Tony GiangrecoCommented:
What Anti-Virus app is installed on the server.  I've seen packet inspection services slow all processes down. This could be your problem. You might want to try disabling the AV. If that helps, reactivate it and look into disabling packet inspection or another function of the AV app that is causing the problem.
joharderCommented:
Based on what you've written, I suspect a network issue.  

First try some basic pings from the RDS to the profile repository.  Try 1500-byte pings.  And a tracert.  Then try them from the profile server back to RDS.

In Network Monitor, you would be looking for retransmissions that would be indicative of a network-related issue.  

Lastly, double check the network config on the RDS servers and profile server.  I once nearly went nuts trying to figure out a similar issue, and it ended up that the issue was an incorrect default gateway config.  That's just something you always *assume* is set up correctly.
kibatsuAuthor Commented:
Hey guys,

I'll add some more information when I get a chance. But I ruled out Anti Virus (AVG), using a test account and the network issues (ping and traceroutes were fine).

I have just found by using the Windows Performance Toolkit that the Search Indexer was running the whole time the logoff process was going (3+ minutes).
On a hunch I disabled the Windows Search service and logoff took under 10 seconds!
I then found out that there was a GPO that is meant to disable file indexing; so I wonder if somehow this GPO and the service still running was causing some sort of conflict during the log off process.

I'll let you know if I find more information - in the meantime, we are disabling the Windows Search service as it isn't something we use anyways.

Cheers guys!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
kibatsuAuthor Commented:
This was quite a unique issue and only by using a monitoring tool was I able to see what may've been causing this.
yo_beeDirector of Information TechnologyCommented:
I am opposed to not give me any points since I recommended that you use Windows Performance Toolkit
David Johnson, CD, MVPOwnerCommented:
yo_bee very indirectly you mentioned WPT only by following a link (you provided) reading and then following a link from it.  I also mentioned WPT but by name and a download link
yo_beeDirector of Information TechnologyCommented:
So you should object to this closure as well since you gave some direction.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.