Solved

Issues with roaming profiles on 2003 TS /citrix environment.

Posted on 2012-03-20
11
1,113 Views
Last Modified: 2012-04-13
Hi Guys, I am trying to resolve a issue that is really doing my head it.  I have a 2003 r2 x86 server that is being used as a citrix presentation server 4.5.  It has hotfix rollup 7 applied. This server has been alive for approx 4 years now and never had a issue.

2 weeks ago multiple times when i log in id see the  following in c:\documents and settings

Username
Username.domain
Username.domain.000
Username.domain.001  etc

Now i have gone through the usual steps of a week full of process monitor/explorer and modifying the LogoffCheckSysModule under the HKLM\System\CurrentControlSet\Control\Citrix\wfshell\TWI.

I am at the stage where now i can see users log in and out with absolutely no remaining processes yet the the profile remains. In the task manager tab it shows the user log in and leave the server. Process explorer shows all processes that started as the user logs in, terminate when the user logs out.  There is no reason i can see why the profile remains.

I have then installed the User Profile Hive Cleaner and am presented with the following errors

The following handles in user profile hive Username (user guid) have been remapped because they were preventing the profile from unloading successfully:
 
winlogon.exe (3492)
  HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings (0x5c4)
  HKCU\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings (0x648)
  HKCU\Software\Microsoft\Internet Explorer\IETld (0x6c8)
  HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings (0x6d0)



Now i have done some research and i have found one of the old IT admins has modified some of the policies in terms of internet settings. Problem is this guy "left against his will" about a week after making these changes.  The problem is i know the changes were not malicious and more likely error in creating the policy.  My problem is i am trying to identify what the code in the (  )  above refer to so i can try and resolve this issue.

Does anyone know what the (0x6c8) , (0x6d0), (0x5c4), (0x648) refer to in regards to the above registry error codes.


I found this support article from microsoft  http://support.microsoft.com/kb/975619 however a call to microsoft and i have been advised this hotfix is no longer valid and microsoft  not willing to provide me with the original file/fix.

Can anyone shed some light into this information for me.
0
Comment
Question by:tetran_au
  • 7
  • 4
11 Comments
 
LVL 23

Assisted Solution

by:Ayman Bakr
Ayman Bakr earned 500 total points
ID: 37741535
Can you check if gpresult or RSOP is functioning properly? I am suspecting a corruption in GPO.

Do you have GPO set to delete profiles when user log off. This setting is Delete cached copies of roaming profiles found in the following GPO node:

Computer Configuration\Administrative Templates\System\User Profiles

Error code 0x6d0 refer to missing OS files. Why don't you try running Windows repair to see if this fixes the issue.
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37746129
Yep, ran a RSOP and came back clean no issues with policies or errors relating to application of policies to users.

GPO is set for delete cached copies of roaming profiles and as mentioned above this has been working fine for a couple years right now.

We have a group of about 15 people all in the same groups and permission set and of this group this issue is only happening to about 5 of them.  The weird thing is i got one of there account details and spent 6 hours logging in via rdp and citrix applications.  I sent emails as the user printed and used pdf however could not replicate the issue.  I logged in and out of the machine approximatly 50 times and no double profiles.  The next morning i logged in again about 4 or 5 times and and no double profiles and its cleaned as it should be.  

The way we know this issue occurs is the users use a particular citrix application that requires email.  The email component is scripted in the usrlogon.cmd to write a certain reg key calling on a required dll key as needed by application.  When this works fine the user can email and login and out without issues with the profile on the TS box.  However as soon as the user gets a duplicate profile on the server the email functionality stops and the user alerts us to the problem.

I have looked for commonalities and symptoms however this seems completely random.  They may be able to log in approximatly 3-5 times over the hours and everything is fine. Then out of the blue this issue will occur where it cant write the profile back.  I have checked following event logs for about 30 minutes before and after this occurs to find some common ground but there is absolutely no logs showing any errors or warnings.  I check the group policy logs for the same and again no signs of errors or not.
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37746368
Ok as part of the investigation i enabled the userenv debug log.

This may shed some light but i will definatly need some assistance disseminating this.

There are 3 sections i am concerned about

1) is about 1/4 of the way thru it where it says

USERENV(210.240) 17:02:22:856 FindTotalDiskSpaceNeeded: Largest 7 file size is 6942286
USERENV(210.240) 17:02:22:856 Available            -1933258752
USERENV(210.240) 17:02:22:856 Needed            7636514
USERENV(210.240) 17:02:22:856 Src size            12779011
USERENV(210.240) 17:02:22:856 Dest size            12779011
USERENV(210.240) 17:02:22:856 Largest hive file            5505024

to me this seems strange that the available space is a negative number. I confirmed that the network share that the profile hive is sitting on has atleast 400gb available left, and that there is no quota settings or restrictions on any way.  Thats the first thing that caught my attention.

2) shortly after the above lines it then starts saying that it is leaving behind various parts of the profile


I havent gone into to much review as yet but attached is the last logoug from levans who logged out at 5:02pm  (17:02)
userenv-debug.txt
0
 
LVL 23

Assisted Solution

by:Ayman Bakr
Ayman Bakr earned 500 total points
ID: 37747742
I have promised my family to go out just right now - so I had a quick look on the file.

I usually would look at the interface where you will find an abrupt change in the time stamp recorded. However, couldn't find such thing.

There seems to be a lot of delnode in the log file. There might be many possible reasons to this (especially that a negative value of available size is recorded):
-> I would think that you might be having network congestions during some time in the day
-> Network file share (where you store your profiles) is hanging during some time in the day
-> I would also definitely look at the size of the profiles of those users having the issues and at their network connections from their PCs.
-> Perhaps try recreating a fresh profile for one of them and test.
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37763593
Hi Mutawadi,

Ok i have narrowerd it down a bit.  I have checked the networking side and the file share and there is no congestion at all. If anything its is currently underutilised.  One of the factors that point away from this is that there is approximatly 30 people on this server at any one time. All 30 have there profiles on the same file share and all 30 inherit 99% of the same policies. Yet it only seems to be the same people day in and day out with the issue (about 8 days and running). That sort of takes the randomness out of it.

Now i have done some more investigation into found some of the original changed made by the employee that is no longer here.

Please have a look at the attached image.  The lines highlighted in the red box's were already there.  The rest of them have been newly added.  Now the User-Opt-AppConfig_MSIE policy is applied to everyone however since not everyone is having the issue i believe that it may just be caught up in the works.

The User-Mand-Flt-AppConfig-MSIE-ANZTransactive policy is the new one.  The image shows the relevant policy applications that sort of relate the error code in my first post. I suspect it may have something to do with the url extensions that have been put in.  

The four users that are having the main issue are all part of a security group that inherits the second policy.  This narrows down my searching a bit more but then raises a few more questions. It seems the second policy mentioned is causing this issue (not sure why yet) but it doesnt answer the question why only 4 or 5 of these users out of a group of about 15 are being affected.  The only other commonality between the affected users is that they are using a windows xp desktop instead of a windows 7.  Being a citrix / ts session with the issue im not really sure how the local desktop could be cauising it.  Both the windows xp and windows 7 are running same version RDP client and exact same version of Citrix Online Plugin.

Confusing issues....  The more i seem to make progress on this one the more questions it is raising and confusing me.

Thanks for the help to date.
policies.png
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37763598
oh and in terms of the sizes of the profiles, they are all under 30mb each so i dont think it is size related
0
 
LVL 23

Accepted Solution

by:
Ayman Bakr earned 500 total points
ID: 37764249
OK. I need to clarify on certain points (I think we are getting somewhere).

1. The policies you are referring to are GPO or local security policies (I guess you are referring to GPOs)

2. When you are applying the policies (if they are GPOs), are you using group filtering (cause you are saying the security group is inheriting the security policies)? Otherwise, those policies are affecting solely the containers they are linked to (OU).

3. Are your windows XP workstations placed in a separate OU from Windows 7 ones?

Can you test with one of those affected users to log on from:

1. A Windows XP machine different from his own - does he get the same issue?

2. A Windows 7 machine - does he get the same issue again?
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37768457
Thanks for the response.  In response to your questions above.

1) Yes they are group policies applied and managed at the domain level.

2) Yes security filtering is applied so only members of this group inherit the changes. The problem only exists with users from within this group and of those users currently only 4 out of 15.  As far as i can see though the policy in question is not applied on the users OU it is applied on the OU where the citrix servers are kept.  Loop back processing is enabled to ensure the correct users get this policy.

3) The windows XP and Windows 7 computers are kept in the same OU
In terms of testing I am 2 steps ahead of you on that.  Testing i have implemented last  night to identify the issue is have one of the affected users from the xp machine swap with a non affected user on a windows 7 machine.  Waiting to see if the problem re-occurs.

Second testing i am completing is removing one of the users from initial security group to see if
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37768467
Just one question thou,

In my first post their is a attachment for policies. is the context in that correct as i see a few different ways. eg

*.anz.com  but then i see microsoft.com and *.microsoft.com  and to add a spanner to the works something like https://*.gov.au.

So above i see 3 different ways that this person has added in the zoning details.  Could the issue be that one or all of these contexts above are incorrect?

I cant just go and make changes to this as i need to justify to the IT manager before making changes. but its a thought that is eating away at me
0
 
LVL 23

Assisted Solution

by:Ayman Bakr
Ayman Bakr earned 500 total points
ID: 37768495
Unfortunately, I don't have an answer to your question regarding the way to represent the site in the policy as I am not sure what is intended with that.

Let's see the results of the testing for the users switched to Windows 7.
0
 
LVL 2

Author Comment

by:tetran_au
ID: 37845696
ok i have finally found the issue I dont know exactly why this is being caused but I have found the problem and the soluton.  I was very close with the gpo's in the first place however i was looking at the wrong one.

I found a URL that was set to be added to the IE trusted zone by GPO.  the url was set by a over eager engineer as

https://*.gov.au  

that this will allow any of the government sites in Australia to be added to the trusted sites list. Unfortunately Microsoft doesn't like adding a top level domain to a trusted list.  Once this was changed to https://*.ato.gov.au etc the gpo errors disappeared both on the domain level as well as the TS level.

Now I am no longer getting any userenv events in the eventvwr and the profiles are no longer getting caught up. Its funny as i found alot of articles about how the urls should be specified in the gpo but could not find many on restrictions relating to it.

Mutawadi thank you for your assistance. I dont think i could have solved this without you as if it wasnt for bouncing idea's off you i dont think i would have persevered for as long as i did.  Hence i am giving you the points. ;)

Hopefully this will help others out there.

Just a heads up

Following allowed context are things like the following

192.168.*.*
*://*.domain.com
http://*.domain.com
*://192.*.*.*
etc

not allowed context

192.168.*
*://*
www.*.*
www.*.com or http://*.net
0

Join & Write a Comment

#Citrix #Internet Explorer #Enterprise Mode #IE 11 #IE 8
This tutorial will walk an individual through the process of transferring the five major, necessary Active Directory Roles, commonly referred to as the FSMO roles to another domain controller. Log onto the new domain controller with a user account t…
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now