New Users Cannot Create Roaming Profile Folder

2016/12/18 malware hits customer network and destroys tons of data and multiple servers – including one domain controller and all of the group policies.

2016/12/18 – 2016/27  I restore or rebuild from scratch multiple servers, restore user data, domain controller, GPO’s etc.

2017/01/03 – Customer is hit again with new strain of similar malware, although updated antivirus had been installed, a user with elevated privileges removed it.

2017/01/04 – 2017/01/08  I again rebuild servers and GPO’s, and restore data, and setup NetLogon debugging, as well as file/folder auditing.

•      No more outbreaks


Pre-existing users who have been configured to connect to their roaming profiles continue to do so.

Any new user created cannot populate new roaming user profiles on existing roaming profile share. All new users get the common “User profile cannot be loaded”.
 

I have duplicated the same AD / file server / security group / user account relationship in my lab and it works perfectly – in fact difficult to break.

I have followed exactly these articles:
https://technet.microsoft.com/en-us/library/jj649079(v=ws.11).aspx
http://www.mcbsys.com/blog/2010/10/reset-roaming-profile-and-folder-redirection-permissions/
http://jeffgraves.me/2013/
http://www.grouppolicy.biz/2010/08/best-practice-roaming-profiles-and-folder-redirection-a-k-a-user-virtualization/

All new users are in exactly the same security groups as the pre-existing users. All new users attempt to connect/create roaming user profiles within the same file server / folder.

To isolate, I moved to brand new workstations, brand new file server, brand new accounts – to rule out anything corrupt on the previous objects. Same exact results.
The new builds were fresh ISO builds, not templates.

I have run packet captures in my lab (where roaming profiles work) and 2 on customer network (one where pre-existing users succeed, where new users fail) and compared them. The successful logons look identical except for a DFS query (DFS is not running so this is curious). The failed logons don’t even appear to try and create the roaming profile folder:

Some of the test workstations were physical machines, some were virtual machines – but I made sure to use new builds of both.
Servers have to be virtual for now.

Checked VMware guest tools, VMware guest versions, NIC types and drivers, etc. All are legit.

Of note:  same users that fail the roaming profile logon can logon (without roaming) and map a drive to the  _EXACT_  location of where their roaming profiles  _should_  be. So permissions don’t seem to be an issue.
I’ve turned up debugging and auditing and the only profile related errors are:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Log Name:      Application
Source:        Microsoft-Windows-Winlogon
Date:          3/6/2017 4:02:51 PM
Event ID:      6004
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      winr2stg.domain.local
Description:
The winlogon notification subscriber <Profiles> failed a critical notification event.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Winlogon" Guid="{DBE9B383-7CF3-4331-91CC-A3CB16A3B538}" EventSourceName="Wlclntfy" />
    <EventID Qualifiers="32768">6004</EventID>
    <Version>0</Version>
    <Level>3</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2017-03-07T00:02:51.000000000Z" />
    <EventRecordID>1476</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>Application</Channel>
    <Computer>winr2stg.domain.local</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Profiles</Data>
    <Binary>F4010000</Binary>
  </EventData>
</Event>


Log Name:      Application
Source:        Microsoft-Windows-User Profiles Service
Date:          3/6/2017 4:02:51 PM
Event ID:      1520
Task Category: None
Level:         Error
Keywords:      
User:          domain\pac.man
Computer:      winr2stg.domain.local
Description:
Windows cannot log you on because your roaming mandatory profile is not available. This error may be caused by incorrect file system permissions or network problems.

DETAIL - The system cannot find the file specified.

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-User Profiles Service" Guid="{89B1E9F0-5AFF-44A6-9B44-0A07A7CE5845}" />
    <EventID>1520</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000000</Keywords>
    <TimeCreated SystemTime="2017-03-07T00:02:51.817433400Z" />
    <EventRecordID>1477</EventRecordID>
    <Correlation />
    <Execution ProcessID="788" ThreadID="1328" />
    <Channel>Application</Channel>
    <Computer>winr2stg.domain.local</Computer>
    <Security UserID="S-1-5-21-2901645698-1785784430-4210207855-17122" />
  </System>
  <EventData>
    <Data Name="Error">The system cannot find the file specified.
</Data>
  </EventData>
</Event>


Log Name:      Application
Source:        Microsoft-Windows-User Profiles Service
Date:          3/6/2017 4:02:51 PM
Event ID:      1500
Task Category: None
Level:         Error
Keywords:      
User:          domain\pac.man
Computer:      winr2stg.domain.local
Description:
Windows cannot log you on because your profile cannot be loaded. Check that you are connected to the network, and that your network is functioning correctly.

DETAIL - The system cannot find the file specified.

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-User Profiles Service" Guid="{89B1E9F0-5AFF-44A6-9B44-0A07A7CE5845}" />
    <EventID>1500</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000000</Keywords>
    <TimeCreated SystemTime="2017-03-07T00:02:51.817433400Z" />
    <EventRecordID>1478</EventRecordID>
    <Correlation />
    <Execution ProcessID="788" ThreadID="1328" />
    <Channel>Application</Channel>
    <Computer>winr2stg.domain.local</Computer>
    <Security UserID="S-1-5-21-2901645698-1785784430-4210207855-17122" />
  </System>
  <EventData>
    <Data Name="Error">The system cannot find the file specified.
</Data>
  </EventData>
</Event>


Log Name:      Application
Source:        Microsoft-Windows-Winlogon
Date:          3/6/2017 4:02:59 PM
Event ID:      6001
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      winr2stg.domain.local
Description:
The winlogon notification subscriber <Sens> failed a notification event.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Winlogon" Guid="{DBE9B383-7CF3-4331-91CC-A3CB16A3B538}" EventSourceName="Wlclntfy" />
    <EventID Qualifiers="32768">6001</EventID>
    <Version>0</Version>
    <Level>3</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2017-03-07T00:02:59.000000000Z" />
    <EventRecordID>1479</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>Application</Channel>
    <Computer>winr2stg.domain.local</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Sens</Data>
    <Binary>F0030000</Binary>
  </EventData>
</Event>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I also had other IT staff double-check my work and they agree it aligns with best practice.

I’ve done this many times in the past and never had this kind of issue before.

There are no mandatory profiles, and never were.

I have created several test servers without any form of antivirus and the results are the same.

I turned on debugging per this article:
https://technet.microsoft.com/en-us/library/jj649075%28v=ws.11%29.aspx?f=255&MSPPError=-2147217396#Events_Step3Enablingandviewinganalyticanddebuglogs
...and it managed to log almost nothing at all.

Thanks in advance for anyone’s insight,
LVL 9
neilpage99Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

neilpage99Author Commented:
Nevermind. I figured it out. For anyone who runs into this problem, check your username. If you follow a naming convention like "firstname.lastname" or "anything.anything" for that matter, you run the risk of having the second part be ".man". In my case I had a username that ended in ".man" (and not my test account either).  Mandatory profiles work off of files that end in .man. Having a username also end in .man creates confusion with regard to roaming profiles. All other forms of usernames worked fine.

Cheers,
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
neilpage99Author Commented:
Found my own answer.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
domain controller

From novice to tech pro — start learning today.