Account Lockouts

No authentication protocol was available.


This line leads me to believe that the workstation could not negotiate an acceptable authentication method (LanMAN, NTLM, ...etc).  In the group policy, do you have SMB signing activated?  This can be found in the Computer-->Windows Settings-->Security settings--> Local Policies-->Security Options.  The options that you would be looking for are "Microsoft network client: Digitally sign communications (always)".  Try to disable this, but realize that this setting stops the "man in the middle" during authentication.

Also try this setting "Network security: LAN Manager authentication level".  This really should be set to "Send LM and NTLM - Use NTLMv2 if negotiated".

also, what service packs are the 2000 DC's at?  If not at SP4, try installing it.  Have you tried removing and re-adding the machine to the domain?  maybe install SP3 on the XP boxes.

I would use Account Lockout Tools to find the issue

The Windows 2000 domain has 6 Windows's 2000 servers that all are at sp4 currently.

I did disjoin both computers and rejoined them to the domain but the issue persisted.

"In the group policy, do you have SMB signing activated?  This can be found in the Computer-->Windows Settings-->Security settings--> Local Policies-->Security Options.  The options that you would be looking for are "Microsoft network client: Digitally sign communications (always)".  

-This is set to disabled currently and the next one "Microsoft network client: Digitally sign communications (if client agrees)" is set to enabled.

"Also try this setting "Network security: LAN Manager authentication level".  This really should be set to "Send LM and NTLM - Use NTLMv2 if negotiated"."
-The current setting for "Network security: LAN manager authentication level" is set to "Send NTLMv2 response only\refuse LM and NTLM." I'm not sure what LM and NTLM and NTLMv2 is all about, but do you think changing this group policy setting to the one you suggested is the way to go. I know the previous network admin was a bit tight with security but I'm not sure I understand all these pieces and whether this might have other effects in the network. Is it worth trying it?

Just some more info from the logs.

I've been combing the DC's security logs and this event happened right before the lockout.

Event Type:      Failure Audit
Event Source:      Security
Event Category:      Account Logon
Event ID:      675
Date:            2/18/2009
Time:            4:34:33 PM
User:            NT AUTHORITY\SYSTEM
Computer:      PDC1
Description:
Pre-authentication failed:
       User Name:      userx
       User ID:            x\userx
       Service Name:      krbtgt/x.ORG
       Pre-Authentication Type:      0x2
       Failure Code:      0x18
       Client Address:      x.x.x.x (the users workstation current dhcp address)


For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

I'm testing the suggested change to the Network Security: Lanmanager Authentication Level as suggested. It should take long to find out tomorrow from the user since this happens 3 or 4 times a day usually.

I do use eventid.net a lot and I have gone through some of the less drastic suggestions. I did download Account Lockout tools and have been trying to use them. I am not sure I understand the whole logon process so interpreting some of the netlogon logs that some of the tools create is a challenge. So far I've only enabled netlogon.log to record a few events since I'm not sure exactly what I'm looking for and I don't want to create a huge log file. I have 150 users and I'm sure the log would fill up quick.

Thanks for the quick response and suggestions. I'll post back tomorrow with what I find out.

Did you create new SIDs for your workstations after you imaged them?  I'd try running NewSID on the ones that are giving you problems just in case it is a SID issue.

Here's a very good article from MS on troubleshooting account lockouts:
http://technet.microsoft.com/en-us/library/cc776964.aspx

Toward the bottom, they list the 0x18 Failure Code as a bad username/password.  It looks like it is a Kerberos authentication problem.  Do all of your workstations have a good time synchronization?

Here: http://support.microsoft.com/kb/885887

Hi getoffmylawn,

The link to request a hotfix I provided above solved my user's who experienced the same exact issue's your user's are. It turned out to be on the Client side, hope it solves your issue as well :-)

Here is the hotfix I downloaded that solved our issues, just rename from txt to exe and execute on the workstations experiencing the issue.

WindowsXP-KB885887-x86-ENU.txt

Thanks for the suggestions. The user that I was testing crokeefe28's suggestion on was not in the office today. So I'll have to wait for tomorrow to be sure.

Paka, I did use newSID on all 50 of the workstations after applying the image and I actually did newSID twice on this users workstation before joining it to the domain to be very sure that wasn't the issue. I'm still new to this workplace and don't want my first project to be my last (ie. building a company baseline). I'll read the site you suggested above as well as reading the KB article on the patch TDKD suggested.

As far as time sync goes, when I look at the workstation's time it is to the minute the same as all the other workstations, servers, and even phone system. The event log show the below event for time sync:
Event Type:      Information
Event Source:      W32Time
Event Category:      None
Event ID:      35
Date:            2/18/2009
Time:            5:00:20 PM
User:            N/A
Computer:      x-034271
Description:
The time service is now synchronizing the system time with the time source dc1.x.ORG (ntp.d|x.x.x.x:123->x.x.x.x:123).

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

So first thing this morning the user was locked out of her account after being logged in for 20 minutes. So I don't think it was the "NTLMv2 only\deny LM and NTLM" setting.

I went ahead and installed the patch that TDKD suggested. Although I wish there was more information in the KB article about it, like what it's changing (so I could check which files it replacing by file version to make sure it got applied properly). But thanks for the file, its much better than installing SP3 which I believe contains the hotfix. I am curious as to what its changing but... A little odd was that the first time I ran the patch it got almost all the way through it before it bombed out with an error that just flashed and went away. I reran the patch and it seemed to go fine the second time. But it makes me think some service that should be running might not be.

Also, thank you Paka for that article. That was what I've been looking for in terms of finding what logging I'm supposed to be doing to trace the problem to the root cause. I was just not able to find it on the microsoft site myself. I've gone though most of the article and enabled the kerberos logging on the client and netlogon logging. So the next time the user has a lockout I should have a lot more information (and some idea of what I'm looking for). Thank you.

So, the patch KB885887 did not work. The user has had another lockout.

Using EventCombMT on the pdc shows the below at the time the account got locked out:
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:46 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:46 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:46 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  10.134.117.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:46 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:46 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:45 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:45:45 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:38 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  someotheruser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.199    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:35 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name: theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:35 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  someotheruser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.199    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:28 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name: theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:27 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name: theuser    User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:25 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  x.x.x.9    
644,AUDIT SUCCESS,Security,Fri Feb 20 09:44:25 2009,NT AUTHORITY\SYSTEM,User Account Locked Out:     Target Account Name: theuser     Target Account ID: %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Caller Machine Name: x-034271     Caller User Name:pdc$     Caller Domain: x     Caller Logon ID: (0x0,0x3E7)    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:25 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name: theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:24 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name: theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:24 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name: theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:24 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.9    
675,AUDIT FAILURE,Security,Fri Feb 20 09:44:23 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  theuser     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-xxxx}     Service Name:  krbtgt/x.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x18     Client Address:  x.x.x.9  


As you can see other users get this as well, but not as many as quickly as the user getting locked out. The failures, I'm assuming they are indicating logon attempts and therefore when they fail it counts against the limit of bad password attempts before the account gets locked out, happen quickly so it would seem to be a process that is doing this and not a user. Is there a way to see what credentials are being used for the mapped drives? Like a command from the command line that will enumerate what network connections are open like netstat -abo or something like that?

Her machine's event logs do have some additional weird events:
Event Type:      Warning
Event Source:      LSASRV
Event Category:      SPNEGO (Negotiator)
Event ID:      40960
Date:            2/20/2009
Time:            9:53:54 AM
User:            N/A
Computer:      x-034271
Description:
The Security System detected an attempted downgrade attack for server cifs/serv1.xdmz.org.  The failure code from authentication protocol Kerberos was "The user account has been automatically locked because too many invalid logon attempts or password change attempts have been requested.
 (0xc0000234)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

and...

Event Type:      Warning
Event Source:      Dhcp
Event Category:      None
Event ID:      1003
Date:            2/19/2009
Time:            8:08:39 PM
User:            N/A
Computer:      x-034271
Description:
Your computer was not able to renew its address from the network (from the DHCP Server) for the Network Card with network address 0022190352B0.  The following error occurred:
The semaphore timeout period has expired. . Your computer will continue to try and obtain an address on its own from the network address (DHCP) server.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 79 00 00 00               y...    

and this is the last event in the security log before the account lockout:

Event Type:      Success Audit
Event Source:      Security
Event Category:      System Event
Event ID:      515
Date:            2/20/2009
Time:            9:40:11 AM
User:            NT AUTHORITY\SYSTEM
Computer:      x-034271
Description:
A trusted logon process has registered with the Local Security Authority. This logon process will be trusted to submit logon requests.
 
 Logon Process Name:      KSecDD

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Have a look here for the various possible troubleshooting
http://www.eventid.net/display.asp?eventid=40960&eventno=787&source=LsaSrv&phase=1 

Are you running an anti-virus package on the servers and clients?  Have you checked all of the services running on the client - most should be running under the Local System account and a few should be under the Network Service account (NT AUTHORITY\NetworkService).  

Since you are running from an  image, I would suspect that the problem has to do with something in the image.  Is it possible to do a regular build of a workstation and have the user that's getting locked out use it?

You might also look at performing a sysprep on one of the images and put it into production to see if that clears the fault.

I have checked all the services and they are all running under System and the Network accounts.

The NIC cards have the most recent driver from the manufacturer's website.

I've checked the cached credentials logged in as the user and its clean.

I have not checked the Time Zone, although the time is set correctly, I will check to make sure they are in the correct time zone.

AV running on both servers and workstations is TrendMicro.

I can do a regular build on a workstation and will try that out next. The problem is it takes time because the user has so many Adobe products, CS3 etc. Adobe's a real pain. And that won't help me find the actual cause. Also, if that does fix it, I don't want to have to go back and reload 50 workstations the long way. But to test it out I can build 1 for the user.

1 additional item of interest might be that I've seen something that mentioned "to many tcp/ip connections". I'm trying to find it again in the logs, but I can't seem to find it again.

Did you try any of the solutions here?
 
http://www.eventid.net/display.asp?eventid=40960&eventno=787&source=LsaSrv&phase=1 

OK...here are few more things to possibly check.......

1.  Any applications that may be using AD credentials (ie:  still logged on to VPN at home, SQL, Phones)

2.  Scheduled tasks on all servers, workstations

3.  Are they logged into multiple locations (test this by only allowing them to login to specific machines on the Account tab within their account)

4.  I know that you said that they were not using cached credentials, but can you disable this in the GPO to be sure?

5.  Finally.....Replication issues.  Are the sites-and-services setup correctly.  Are the subnets configured in AD to map to a specific site.  Is replication from the 2000 servers to 2003 working.  Check using REPLMon ro "repadmin /showreps"

Also, mapped drives on another machine would cause these errors

Here is a way to do further troublshooting....  The failure codes that you are listing are different.  Some are for lockout issue and some are for account restrictions.  here is a table for the failure codes listed in your event logs:
Check the account restrictions for login time violations, workstation violations.

Error Code

Cause 6
The username doesnt exist.

12
Workstation restriction; logon time restriction.

18
Account disabled, expired, or locked out.

23
The users password has expired.

24
Pre-authentication failed; usually means bad password

32
Ticket expired. This is a normal event that get frequently logged by computer accounts.

37
The workstations clock is too far out of synchronization with the DCs clock.

How about this hot fix  http://support.microsoft.com/kb/887572
 
A scheduled task that is running under a specific account cannot access a shared network resource in Windows XP

The Time Zone was set correctly, so that was not the issue.

Any other ideas or suggestions while I build a new workstation not from the image?

You could also take the PDCE out of the equation by transferring that role to another 2003 DC.  Attach to the DC you want it be on, Right click your domain in ADU&C, click Operations Masters, and change the PDCE role.

I submitted 4 additional comments above

dstewartjr - I tried some of the suggestions, specifically, I tried:
-The network cards have the latest drivers on both workstations and servers.
-[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon]
"GpNetworkStartTimeoutPolicyValue"=dword:000000b4
-TimeZones are correct and Time sync works and is correct.
-Users do not use Terminal Services.
-The Client for Microsoft Networks is present.
-The NIC is not allowed to go to sleep.
-I tried the permissions on these keys: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Dhcp and HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip
-I tried adding this reg key for UDP packet fragmentation:HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\LsaKerberos\Parameters\MaxPacketSize=1

I am curious about the DNS solutions since I do see warning about DNS in their event logs, specifically, the suggestion to:One common service/server mentioned when this event is recorded is DNS/prisoner.iana.org. This DNS server, "prisoner.iana.org" is one of the RFC 1918 "blackhole" servers setup to answer requests related to private IP addresses (RFC 1918) like 192.168.0.0 or 10.0.0.0 that normally should not go out on the Internet for resolution. Most probably, one service running on the local computer is trying to resolve the host associated with an private IP address but the local DNS server is not configured with a reverse zone for this private block of IPs so it sends the request on the Internet root servers (and from there redirected to the bogus prisoner.iana.org). So this event is caused by a misconfiguration of your network. To resolve this issue create the proper reverse lookup zones for the private IP subnets used on your network."

The only problem is the resolution is not very clear. I'm not sure what "proper reverse lookup zones" are, one of the down sides to relying completely on AD integrated DNS is that I haven't had to deal with DNS much.

Could this account logout be due to a DNS problem?

crokeefe28,
-The mapped drives on the new workstation are all from the login script and use the users authentication and the user has the proper permissions to all the shares and can usually access the drives, unless her account is locked out. The old machine was not disjoined from the domain, but it was deleted from AD.
-Currently the user has no workstation restrictions or logon time restrictions in AD, so I'm not sure why 12 comes up. And if I'm reading the log right, 18 comes first which means the account is locked out and then 12 comes up (which it shouldn't at all because there are no restrictions). I'm a little hesitant to put the machine restriction on to test because she does need to be able to use her mapped drives so would I have to add the file server or any other box she needs access too, or just her workstation?
-And I am bring a new PDC online within the next week or 2 and will be transferring all the FMSO roles at that time, so I'm going to wait on your last suggestion.

Your other suggestions:

1.  Any applications that may be using AD credentials (ie:  still logged on to VPN at home, SQL, Phones)
-The user does not use VPN. We use Avaya MM so the phones do have a connection, and she does use a Access front-end with linked tables to a SQL db in a 1-way-trust domain (other domain). But the ODBC connection for this seems fine and lots of other people in the building use this as well without issue. She does use OWA from home a lot but there is no way in Exchange to close the OWA connections and TCPView doesn't make clear who is using what connection. Does Outlook cache credentials? We also have some sites that use AD credentials picked up from the logged on users. Other than that there's nothing I can think of.

2.  Scheduled tasks on all servers, workstations
-This is disabled by group policy on the workstations. Servers only use service accounts for scheduled jobs. (I create all the jobs.)

3.  Are they logged into multiple locations (test this by only allowing them to login to specific machines on the Account tab within their account)
-Would I also have to add machines they connect to via network drives (file server, etc.)?

4.  I know that you said that they were not using cached credentials, but can you disable this in the GPO to be sure?
-The reason I was sure was because it is disabled in the GPO. And I double-checked by making them admins, logging in and manually checking.

5.  Finally.....Replication issues.  Are the sites-and-services setup correctly.  Are the subnets configured in AD to map to a specific site.  Is replication from the 2000 servers to 2003 working.  Check using REPLMon ro "repadmin /showreps"
-Using replmon all the replication seems to be working just fine. In fact, as I mentioned above, I promoted 2 DC's today and they replicated rather quickly with the exiting DC's without any errors.

So, I would like to make sure DNS is setup properly the reverse lookup zones are correct. When I use LockoutStatus.exe from ALTools the local DC in the Default-first-site shows user state as locked and bad pwd count as 5 and last bad pwd from when the lockout occurred, but the bdc in the secondsite shows bad pwd count as 0 and last bad pwd from a month ago. It just looks odd.

Build a test machine not from the image and deploy to 1 of the 2 affected users to test whether something in the image is creating problems.

Put aloinfo.exe on 1 of the 2 workstations so I can post the log to see if anyone here could decipher its meaning, because I have no idea what it means. I attached it because its kinda long at 282kb.


alockout.txt

Try this, goto control panel(on xp client)>>>user accounts>>advanced>>manage passwords>>clear stored passwords

this is another way
 
at a command prompt type
control keymgr.dll
This brings up a "Stored User Names and Passwords" dialogue.
You can remove the locally cached passwords.
 

dstewartjr - I tried to clear the managed passwords while logged in as the affected user (who i gave temporary admin priviledges) and the clear stored passwords button was grayed out, which i took to mean there were none. I will try the keymgr.dll way logged on as the local admin (I don't know the users password and they are gone for the day).

Did you look at the alockout.txt file from aloinfo.exe?  I'm curious if there is any good information in there.

Yes, that's what lead me to a link with those suggestions.
 
specifically this line
 
***WNetUseConnectionW Failed!*** (1), Local: (null), Remote: \\, Password: Password was NULL

I'm not sure what the is talking about because that drive is set in AD in the User's account as their Home Directory.

Early on when trying to diagnose this initially I ran netdiag on the workstation and everything passed but I saw this and am not really sure what it means. Any ideas?

NetBT name test. . . . . . : Passed
[WARNING] At least one of the <00> 'WorkStation Service', <03> 'Messenger Service', <20> 'WINS' names is missing.

I ran the control keymgr.dll and it was still the same. Everything was grayed out and there was nothing listed.

Are you using exchange?

Sorry, the workstation service is running and set to run under the system account, messenger is disabled (but that shouldn't hurt anything), and i don't know what wins would be called but don't see that one in there.

Yes, Exchange 2003 sp2, and now that you mention it, it has been getting patched since Nov with all the patches.  That includes that stupid Send As patch. Why do you ask?

saw something that earlier that mentioned using same mailbox password, But now I think it is more related to being an image.
 
Did you run newsid on these? also try unjoining and rejoining to reset the machine account password

you can use netdom too
http://support.microsoft.com/kb/216393 

Another good possibility is that you may have a rootkit problem

RootkitRevealer v1.71

I did run newsid on these after they were imaged and before being joined to the domain. I have tried unjoining and rejoining, but can try it again. I'm not sure what you mean my "machine account password"?

I'm really hoping its not the image. Do you think it could be Exchange related. Like they had permissions to view someone's calendar and contacts and because of some update when their Outlook tries to pull that information from some other users mailbox its getting access denied?

FIrst look into the rootkit

You mean run rootkitrevealer on it or something like that?

yes, in meantime I asked another expert to take a look at what you have going on.

Hi,

Go into Exchange system manger and remove relay permissions for all authenticated users.  

Start ESM, expand admin groups, servers, the server, protocols, smtp, right click on the smtp virtual server, properties, click on access tab, click on the relay button, select users at the bottom and then deselect relay permissions from authenticated users.  There shouldn't be anything else in there.  If there is then it is customized and let us know.  Click apply and ok back to the ESM and right click the smtp virtual server and stop and start it.  Check your queues.  See if there are a ton of queues that your users wouldn't be emailing to.  

If you are being locked out then this is more than likely already unchecked, being that you should receive some user could not authenticate 1709 and 1720 events.  Either way, you should do this as it is best practice.

If the relay is unchecked then this is what is causing your accounts to lock out.  They are more than likely infected with trojans or rootkits that are try to spam through your email server as an open relay.  When unchecked, your accounts will not authenticate and will automatically lockout.

Other than that, what has been described above is sufficient to troubleshoot.

HTH

Also, reset your domain's administrator password and ensure that guest is disabled.  Look through ADUC and check high level accounts for permissions that they shouldn't have, especially in the builtin users OU.  Look for anything out of the ordinary, and be sure to check the domain admins group for legitimacy.  

Thanks Mighty, Exchange not my forte' .....yet

The relay was unchecked. But if I'm following you I should see events 1709 and 1720 somewhere if it is a rootkit problem. So where would I see those events, on the exchange server or on the workstations?

on the DC.  You will see it under the security events.

Guest is disabled. And the domain admin group is legit. I will check the users ou and change the domain admin password.

Also run the root kit check on one of the workstations that is getting locked out.

RootkitRevealer v1.71

I'm checking all the DC's for the event id's you mentioned. If those events don't show up then...? Run the rootkitrevealer anyway, it can't hurt. But where does that leave me?

If your relays were unchecked already for authenticated users then this REALLY points to your lockouts.  If you see rootkits then you will have to do an SMTP cleanup and clean ALL of your workstations.  Lets hope that this is not the case.  This would obviously be a rather large scope.  

Again, if you run a root kit checker on a workstation that is normally locking out and you find something then at least we know the cause.

There are no events 1709 or 1720 on the DC's. I ran rootkitrevealer v1.7.1 and the results are posted below. There isn't anything in them that I can see.

HKLM\SECURITY\Policy\Secrets\SAC*      10/14/2008 12:30 PM      0 bytes      Key name contains embedded nulls (*)
HKLM\SECURITY\Policy\Secrets\SAI*      10/14/2008 12:30 PM      0 bytes      Key name contains embedded nulls (*)
HKLM\SOFTWARE\TrendMicro\PC-cillinNTCorp\CurrentVersion\Misc.\TotalScanned      2/20/2009 7:02 PM      4 bytes      Data mismatch between Windows API and raw hive data.
HKLM\SOFTWARE\TrendMicro\PC-cillinNTCorp\CurrentVersion\Misc.\LastScannedFileName      2/20/2009 7:02 PM      61 bytes      Data mismatch between Windows API and raw hive data.

That is a good thing then.  What about scheduled events on the computers, go under windows/tasks and see what is running and why.  Also, is outlook trying to authenticate during off hours?  Do you have prohibited hours setup anywhere for their accounts?

You scared me for a minute there.

Scheduled tasks are prohibited on workstations by GPO. Plus I checked these 2 machines specifically and there is nothing there. And there are no prohibited hours setup for their accounts, either workstations or logon hours. I'm not sure about Outlook trying to authenticate during off hours. But the account lockouts only happen while the users are logged into their workstations and working. I work most weekends and for a month I've been checking and I have yet to see an account lockout when they are not in the office sitting at their desks with Outlook, Excel, Access, and 15 web pages open.

see if this script gets more info. provide wslist.txt with computernames

'==========================================================================
'
' NAME: getLockoutLocation.vbs
'
' AUTHOR: Mark D. MacLachlan , The Spider's Parlor
' URL: http://www.thespidersparlor.com
' DATE  : 1/4/2004
'
' COMMENT: <comment>
'
'==========================================================================
 
On Error Resume Next
 
'open the file system object
Set oFSO = CreateObject("Scripting.FileSystemObject")
 
'open the data file
Set oTextStream = oFSO.OpenTextFile("wslist.txt")
'make an array from the data file
RemotePC = Split(oTextStream.ReadAll, vbNewLine)
'close the data file
oTextStream.Close
report = ""
 
For Each strComputer In RemotePC
    Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")
    Set colItems = objWMIService.ExecQuery("Select * from Win32_UserAccount",,48)
    For Each objItem in colItems
        If objItem.Lockout = "False" Then
            report = report & "User: " & objItem.FullName & " is locked out on " & strComputer & vbCrLf
        End If
    Next
Next
 
 
Set ts = oFSO.CreateTextFile ("LockoutLocation.txt", ForWriting)
ts.write report
Set oTextStream = nothing
set ts = nothing
set oFSO = nothing

Select allOpen in new window

You want me to run this on the affected workstation? Logged in as admin or as the user? Before or after a lockout? Does it matter?

I ran it as the local admin on an affected workstation and it generated a blank file. So, I'm guessing run it after a user gets locked out on the locked out workstation...?

Just as an aside, I have seen some weirdness in this domain before. There was a program in a different department that when installed created a com object that needed to run under different credentials. The program would work until the workstation was rebooted (and the group policy reapplied) at which time the credentials for the com object would change. An admin would have to logon and reset the credentials for the com object and then it would run properly until the next reboot. So I tweaked the GPO's slightly when rolling out this new image and the program runs properly now (the com objects credentials aren't reset by the group policy being reapplied). I'm not exactly sure what it was in the group policy that I changed that allowed this program to work, but I took the approach of only setting GPO settings that I needed set for company policy or was considered best practices (or were already set in the old GPO and I wasn't sure what they did). I could have inadvertently done something weird in the GPO settings, but what it might be I'm not sure because only 2 of 50 users are adversly affected and the 2 don't seem to have anything in common (2 different departments with different programs that don't have any unique programs that others with newly imaged workstations with the new GPO's don't have. The only exception to that might be some departmental Access db's. The users affected by the lockouts did not have this problem on their old computers before I gave them new computers and moved them to the new computer and user GPO's.  If it helps I can post a list of services and will email any requested sections of GPO's that anyone thinks might be relevant.

How is the new build coming along?  This will certainly help localize the cause of the lockout.  Just make sure you start with a pretty clean xp build and only layer Office on top to start.  Next, I'd recommend joining it to the domain and setting up an Outlook profile and turning it over to the user.  If their account is locked out with the minimum install, then it's likely a GPO, script, or server end problem.  Next, layer on the Adobe items and give it an hour of run time.  Finally, layer the extras (like the Com+ app if you're using it).  A new build might seem like a heavy investment in time, but it will help localize client end from server end.

Do you have any other specialized apps (like the Com+ one mentioned earlier) running in your environment?  How about web-based authentication apps?

No other specialized apps besides the one I mentioned and only a few people in 1 department use that app. None of the affected users use it.

There are many web-based apps, I'm not sure I know them all. Both these users use 1 in particular that I know of. But it's not AD authentication that I know of. They have to log into the app hosted by an outside vendor that pulls data from one of our inside SQL backend db's...

I got a bit distracted from the build today, was moving servers between racks and replacing UPS's. But by Monday I should be able to give it to the user to test out, with a minimal build.

Thanks.

Just on the outside chance... Does the SQL backend use SQL or NT authentication?  Is the vendor still able to access the SQL backend?

I asked that same question many posts ago, but he/she informed the following:

she does use a Access front-end with linked tables to a SQL db in a 1-way-trust domain (other domain). But the ODBC connection for this seems fine and lots of other people in the building use this as well without issue.

I thought that they may be using cached credentials of some sort.  Could always double check the SQL logs as suggested before

The SQL backend uses NT Authentication. The ODBC connection uses her NT credentials.

Sorry about the delay, but I deployed a new workstation and there were no account lockouts for 1 day, a full 8 hours at work. Then the next day the account lockouts started again. Thinking that it now had to be a group policy setting I moved her machine to a different OU with a different GPO and her user account to a new OU with a different GPO. (And rebooted the machine twice). These are the older GPO's that both uses have used without any problem since I've been working here. Everything was okay for most of the day and then the lockouts started up again. Yesterday and today the lockouts were happening every 10 to 60 minutes until noon. At noon they stopped. No more lockouts for the rest of the day. The user has had the same 10 windows open all day (4 Internet Explorer instances, 2 Firefox instances, 2 excel spreadsheets, Outlook, and an Access db). The user had these windows open at 9:10am when the first lockout occurred and had them open after noon until they left at 5pm. I've had procmon.exe running all day and have a log file of about 2gb and 7million events. I saw a couple of weird things, like winlogon.exe calling a wgalogon registry key...? Wndows Genuine Advantage was on the computers, but was removed so imaging would work. There are some ACCESS DENIED events for winlogon.exe and lsass.exe. However, if I filter the log to only the exact time that each lockout occurred today, the only thing they all have in common is Internet Explorer getting ACCESS DENIED events. I'm not sure where to go from here. We run IE6 because IE7 does not work with some of our programs. (Hence the Firefox install, cause Firefox just works with everything). I feel like asking the user not to use IE for a day, but with the weird infrequency of the lockouts... I'm also thinking that maybe installing XP SP3 might be a next step.

Any suggestions?

try this:

http://kb.iu.edu/data/alcx.html

Internet authentication will have nothing to do with windows authentication.

I dont think that you are using the account lockout tools fully


AcctInfo.dll. Helps isolate and troubleshoot account lockouts and to change a user's password on a domain controller in that user's site. It works by adding new property pages to user objects in the Active Directory Users and Computers Microsoft Management Console (MMC).

ALockout.dll. On the client computer, helps determine a process or application that is sending wrong credentials. Caution: Do not use this tool on servers that host network applications or services. Also, you should not use ALockout.dll on Exchange servers, because it may prevent the Exchange store from starting.

ALoInfo.exe. Displays all user account names and the age of their passwords.

EnableKerbLog.vbs. Used as a startup script, allows Kerberos to log on to all your clients that run Windows 2000 and later.

EventCombMT.exe. Gathers specific events from event logs of several different machines to one central location.

LockoutStatus.exe. Determines all the domain controllers that are involved in a lockout of a user in order to assist in gathering the logs. LockoutStatus.exe uses the NLParse.exe tool to parse Netlogon logs for specific Netlogon return status codes. It directs the output to a comma-separated value (.csv) file that you can sort further, if needed.

NLParse.exe. Used to extract and display desired entries from the Netlogon log files.

dstewartjr, i agree with you. However, i posted a log from alockout.dll earlier and didn't get much out of it. It is currently installed on both affected workstations. Nor did anyone else from what I can tell. AcctInfo.dll is installed on my workstation but it doesn't provide any information on what is causing the lockout, just when it happens. I don't see how aloinfo.exe is helpful in this case. EnabelKerbLog seems like a good idea and I thought it was a run once thing, not a startup script. I will take your advice with that and use that. EventCombMT is great and I have been using it to gather log files. Specifically the built in search for Account Lockout events is great. And LockoutStatus.exe I have not used and probably should.

I was forced to create a new account for the user last night. I created a new account and gave the new user account rights to the old email address to save myself some work. Then I logged onto the users affected machine this morning and setup the new account (email, fonts, that junk). Interesting thing I just noticed. The user has been logged into the new account all day with no lockouts, but her old account was unlocked this morning at 9am but became locked at 11:10am. Noone should be logged in under that account right now. How could the account become locked out?

There's gotta be something that the account is trying to authenticate against.
 
Try this
Download currports and then log into that machine with the old account and run currports to see if you see anything out of the ordinary.
http://www.nirsoft.net/utils/cports.html 

couple other things to try:
 try running psloggedon.exe to enumerate all machines the faulty user is logged onto http://www.ss64.com/nt/psloggedon.html
http://live.sysinternals.com/psloggedon.exe
look for  event id 12294  

Event ID 12294  Account Lockout

This can also be your issue
 
Troubleshooting Account Lockout
 
Bad Password Threshold is set too low: This is one of the most common misconfiguration issues. Many companies set the Bad Password Threshold registry value to a value lower than the default value of 10. If you set this value too low, false lockouts occur when programs automatically retry passwords that are not valid. Microsoft recommends that you leave this value at its default value of 10. For more information, see "Choosing Account Lockout Settings for Your Deployment" in this document.
 

The threshold is set to 5 which is a company policy and was revisited not to long ago by the legal department. That is not going to change anytime soon.

psloggedon came back with nothing. I ran it again and sent the output to a txt file and searched for the username and still came up with nothing.

I'm going to run a couple of eventcombmt scans to see if I can find where the lockout for that account was coming from. Then I will run tcpview and currports on that machine.

Paka Try reading the other comments
 
"I was forced to create a new account for the user last night. I created a new account and gave the new user account rights to the old email address to save myself some work. Then I logged onto the users affected machine this morning and setup the new account (email, fonts, that junk). Interesting thing I just noticed. The user has been logged into the new account all day with no lockouts, but her old account was unlocked this morning at 9am but became locked at 11:10am. Noone should be logged in under that account right now. How could the account become locked out?"

crokeefe28, I tried http://kb.iu.edu/data/alcx.html and it didn't work. The users account was locked out again within 10 minutes.

I ran currports and the log file is attached. The last account lockout was at 2:55:13pm.

Paka, I have created a new user account for 1 of the users and that new account, which the user has been using all day has not locked out at all, however, the user's old account continues to lockout even though noone should be using that account. I would really like to know the cause of this because I'm sure it will come up again in the future. Plus, the users have gone days without a lockout and then it starts happening again.
lockedout.txt

675,AUDIT FAILURE,Security,Fri Feb 27 2:41:47 2009,NT AUTHORITY\SYSTEM,Pre-authentication failed:     User Name:  ajamima     User ID:  %{S-1-5-21-52832475-1979476044-1722246644-1112}     Service Name:  krbtgt/BAR.ORG     Pre-Authentication Type: 0x2     Failure Code:  0x12     Client Address:  10.134.117.17    

I see that event in the logs using eventcombmt a lot for this user's old account all day and the IP indicates that its coming from her current workstation, which I did replace right after this started happening. And the fact that she's using a different account on the same machine without lockouts has me really confused. What would be running on the computer with her old user credentials if she's not logged into the computer under her old account?

You said you checked for services that may have this users old credentials, try checking if dcom to see if any are configured with this user

What do your event id 539 reveal ??
 
Note: The Logon Types are:
Type 2 : Console logon - interactive from the computer console.
Type 3 : Network logon - network mapping (net use/net view).
Type 4 : Batch logon - scheduler.
Type 5 : Service logon - service uses an account.
Type 7 : Unlock Workstation.
Type 8: NetworkCleartext - Logon with credentials sent in the clear text, for example logon to IIS with the basic authentication.
Type 9: NewCredentials.
Type 10: RemoteInteractive - Logon remotely such as Terminal Services, Remote Desktop or Remote Assistance.
Type 11: CachedInteractive - Logon with cached credentials, for example logon a work laptop at home or using VPN)
Type 0 & 1 are not used and Type 6 is listed as a proxy.

Using EventComb on the DC's and workstations looking for Event ID 539 I see only type 2 and Type 7, and they are not associated with the timing of the account lockouts.

I also see Type 3 for that workstation at like 5:22pm yesterday, which is a weird time for a drive mapping.

That DCOM check is going to take a while with pointing and clicking, but I'll let you know once I'm done.

yeah, it might have been reaching but running out of ideas :-)

First, the event times for each Audit do not have to match up because AD increments the badPwdCount by one each time you attempt.  it passes this information transitively up the stream to the PDCE for record keeping.  

Not sure if this has been done (after all the posts) but let's try enabling a little more logging on the client side:

Click Start, click Run, type regedit, and then press ENTER.
Add the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters registry value to the registry key:

Registry value: LogLevel
Value type: REG_DWORD
Value data: 0x1
If the Parameters registry key does not exist, create it.
Close Registry Editor and restart the computer

Also, let's dive into the GPO a little farther (once again, not really sure that we have done all of this) and look into the setting for "account lockout".  

1.  What is the bad password increment
2.  What is the reset count policy
3.  What is the policy to lock an account out
4.  How long?
5.  How about on the 2000 domain?
6.  What is the Kerberos  sensitivity to time +/- how many minutes within the Policy
7.  Have you check the netlogon.log from the PDCE and traced it back to the offending, originating server?
8.  You mention this 2000 domain.  What type of trust?
9.  can you enable "Audit Process Tracking" for the domain within the GPO under Audit Policies
9.  Looking at the logs that you sent in the posts, it does seem as though there is sometimes a 10 minute difference       from the client to the server.  Are you sure that it is correct all the way around?


Enable debugging on the netlogon:

This creates a text file on the PDC that can be examined to determine which clients are generating the bad password attempts.

http://support.microsoft.com/kb/189541
http://support.microsoft.com/kb/109626

The benefit of copying the user account is to eliminate it from any service account or drive mapping issues that might be associated with that account.

I checked the DCOM objects and 94 are set to "The launching user", 23 are set to "The system account", and 17 are set to "Interactive User", 0 are set to use the given credentials.

I did see this event in the log which also seems to indicate that a service is running under the user account:
Event Type:      Warning
Event Source:      Userenv
Event Category:      None
Event ID:      1517
Date:            2/25/2009
Time:            8:27:50 PM
User:            NT AUTHORITY\SYSTEM
Computer:      BAR-034391
Description:
Windows saved user BAR\ajamima registry while an application or service was still using the registry during log off. The memory used by the user's registry has not been freed. The registry will be unloaded when it is no longer in use.

 This is often caused by services running as a user account, try configuring the services to run in either the LocalService or NetworkService account.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

I'm not sure where I would find the increment, but the domain pwd policy is max age 90 days, min age 1 day, lockout duration 2 hours, reset bad pw count 1 hour, max bad pwd count is 5, previous kept 15, min pw length 9 characters. The 2000 domain has the same policy. Kerberos time sensitivity is 3 minutes +-. The trust between the 2 domains is a 1 way trust with the 2000 domain trusting the 2003 (our) domain. I had put that kerberos logging reg key in previously with a value of 1 (decimal) but don't really see anything extra in the the event logs, is there somewhere else I should be looking for that kerb log? I have not got the netlogon.log configured correctly yet to capture the events I need. I will fix that. I temporarily enabled Audit Process Tracking for Failure.

Event Source: Userenv
Event Category: None
Event ID: 1517
Description:
Windows saved user ComputerName\UserName registry while an application or service was still using the registry during log off. The memory used by the user's registry has not been freed. The registry will be unloaded when it is no longer in use. This is caused by services running as a user account, try configuring the services to run in either the LocalService or NetworkService account.
The solution was in this knowledge base article KB837115: Troubleshooting profile unload issues. Microsoft provides a service you can install called UPHClean that monitors the computer while Windows is unloading user profiles and forces resources that are open to close.

Have you checked to see if there are any time differences between the 2000 domain and the 2003 domain?  W2000 uses SNTP (which allows a clock drift of 5 minutes within the domain); 2003 uses NTP which usually keeps time drift to less than a second.  It's a stretch, but that Windows 2000 5 minute drift tolerance might be triggering a Kerberos fault (although you should see evidence in the event logs).

Paka, the Windows2000 domain is within 10 seconds of a timediff with the Windows2003 domain.

I do see this user and other users getting Kerberos pre-authentication errors in the security event log on the domain controllers but only 2 of about 10 users who are getting these Kerberos pre-authentication errors are having this lockout problem. I posted earlier what those errors were and I'm not sure how "pre-authentication" fits into the authentication process since I don't see any other errors except after the user account gets locked out.

Have you installed UPHclean?
 
Also have you read?
 
Account Lockout Best Practices White Paper

That is the reason for the netlogon.log.  We should be able to follow the error downstream from the PDCE to find the offending machine.

It snowed on the East Coast this morning and most of my users are still at home. I'm not sure if the 2 affected are in the office today. I will install UPHclean on both.

I've read part of the White Paper but not the whole thing. I will at lunch today.

So the netlogon.log should show the authentication attempt from the users workstation against the server with the wanted file/service through the PDCE? That would help because so far I only have the workstation and the PDC in the equation and I'm sure something is missing.

The log will show where the request came from as all request from server authentication go upstream finally hitting the PDCE.  You can follow the stream to see where it originated from....match up the times and HOPEFULLY voila.

Snowed here too :-)
 
from the whitepaper
 

Possible Solution: If you determine that the log files display a specific number of logon attempts for a each user, add the number of occurrences of 0xC000006A and 0xC0000234 errors for the user. In some scenarios, you may see a pattern of a specific number of attempts for one user, and then the same number of attempts for another user, and so on. This behavior may be an attack on the network or a program could be sending a specific set of attempts. 16 or 17 attempts per user is a common figure for these types of attacks.
In most account lockout situations, you must use Netlogon log files to determine which computers are sending bad credentials. When you analyze Netlogon log files, look for the 0xC000006A event code, because this event will help you determine where the bad password attempts began to occur. When you see the 0xC000006A event code and it is followed by a 0xC0000234 event code, the event codes that come after these event codes help you determine what caused the account lockout. If you see patterns in the log files, the patterns can help you determine if the event code was logged because of either a program attack or user error.

I'm trying to get netlogon.log going and am a little confused. The DC's are both WinServer2003sp1 and the White Paper indicates that you can set netlogon logging from the command line for Win2000 servers and you can use lockoutstatus.exe as well. When I right click on the DC in lockoutstatus.exe I get a menu for Netlogon Logging with options for Initialization, Logon Processing, Misc Debug, Synchroniztion & Replication, Mailslot Messages, Sites, Critical Events, etc. I selected Logon Processing and when I look in the netlogon log file I only see a bunch of [MISC] events like the ones below. If i try to click all the options just to get the most information when I go back and check the settings they have reverted to only a couple of items. What should be checked so that I will get the logon information I need to troubleshoot this?

03/02 12:23:10 [MISC] BAR: NetrLogonGetDomainInfo: DnsHostName of BAR-034281 is BAR-034281.BAR.ORG
03/02 12:23:19 [MISC] BAR: DsGetDcName function called: Dom:(null) Acct:(null) Flags: DS BACKGROUND

you can use the command line
 
nltest /dbflag:0x2080ffff
 
 Enabling debug logging for the Net Logon service

Interesting. I ran the command on all 3 of the DC's and on 2 it went fine. The third it said it could not find nltest as a legit command...? Odd. Both older DC's are slightly quirky (like on one AD looks weird in the mmc  - everything expands on the left and nothing populates on the right hand side) but both are up and working and the 3rd one is one of the replacement's that I brought online last week. Their quirkiness and being sp1 are a couple of reasons I'm in the process of replacing them. Anyway, neither user with the issue is in the office today, but now that the logging is enabled (and I do see [NETLOGON] events now) hopefully some more helpful information will come to light. Thanks.

So, the account lockouts continue. The lockouts today occurred 11:21:25am, 12:14:41pm, 1:49:45pm. I looked through the netlogon.log (using nlparse.exe) and the the events that identify the workstation and the username are below (I don't see anything out of the ordinary because the C234 seems to be after the lockouts occur):

3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:18      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:22      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:22:22      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      11:36:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via BARCSDC036579)      0x0
3-Mar      12:14:55      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:14:55      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:14:55      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:14:55      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:14:57      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:14:57      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:15      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:15      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:25      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:35      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:35      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:15:35      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:16:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:18:59      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:19:01      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:19:01      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:19:01      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:21:15      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:21:15      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:21:23      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:21:23      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:23      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:23      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:23      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:27      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:27      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:43      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:43      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:43      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:43      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:43      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:22:44      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:03      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:03      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:05      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:09      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:33:09      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:46      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:46      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:46      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:47      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:51      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:51      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:52      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:44:53      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:50:14      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:50:15      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      12:50:15      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0xC0000234
3-Mar      13:11:29      SamLogon: Network logon      BAR\ajamima      BAR-034391      0x0
3-Mar      13:11:29      SamLogon: Network logon      BAR\ajamima      BAR-034391      0x0
3-Mar      13:40:56      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0x0
3-Mar      13:40:56      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0x0
3-Mar      13:40:56      SamLogon: Transitive Network logon      BAR\ajamima      BAR-034391 (via PRINTSERV1)      0x0
-------------------------------------------------------------------------------------------------------------------------
3-Mar      1:01:28      SamLogon: Generic logon      BAR.ORG\(null)      (null) (via BAR-034391) Package:Kerberos      0x0
3-Mar      2:49:28      SamLogon: Generic logon      BAR.ORG\(null)      (null) (via BAR-034391) Package:Kerberos      0x0
3-Mar      4:32:28      SamLogon: Generic logon      BAR.ORG\(null)      (null) (via BAR-034391) Package:Kerberos      0x0
3-Mar      6:26:29      SamLogon: Generic logon      BAR.ORG\(null)      (null) (via BAR-034391) Package:Kerberos      0x0
etc. there are more of these machine null logons throughout the day. The null thing seems weird to me but I'm not sure if its normal or not.

I did install UPHClean on both computers before the users logged on this morning, and rebooted the machines. I installed it under the local admin account because nothing said it had to be installed under the users account.

I ran EventComb against the DC's and the workstations and did notice this event below. Logon Type 11 is cached credentials if I'm not mistaken. This would seem to validate that the System account is using the users old cached credentials for some reason to do something. Now figuring out what it is...?

529,AUDIT FAILURE,Security,Tue Mar 03 09:25:29 2009,NT AUTHORITY\SYSTEM,Logon Failure:     Reason:  Unknown user name or bad password     User Name: ajamima     Domain:  BAR     Logon Type: 11     Logon Process: User32       Authentication Package: Negotiate     Workstation Name: BAR-034391

The description for 0xC0000234 is "The user account has been automatically locked". And like I said the events in netlogon correspond to right after the user's account was locked out and before I unlocked the account. When I look at the users computer with the user logged in but the account locked out, I can see in Printer and Faxes that none of her printers are available because her account is locked out. So if she tries to print while her account is locked out she can't (and I would think this C234 event is logged because the print server is checking to see if the account has permissions).

I guess I'm more confused by what I don't see in the netlogon.log file, like where is her logging into her computer first thing in the morning, or when she comes back from lunch (password protected screensaver goes on after 25 minutes of inactivity). I put every netlogon event that used her name above and I don't see any of that stuff. That plus the one event of type 11 makes me think that cached credentials are being used and that I messed something up in the group policy. For instance, in the GPO under Computer Configuration -> Security Settings -> Security Options -> Interactive logon: Number of previous logons to cache (in case domain controller is not accessible) = 2 logons. That was they way the old GPO for the client computers was set so I just copied it to the new GPO. So, just as a test I copied the GPO and changed this one setting to 0 logons and put the user's computer in the GPO for the day.

Don't get me wrong I'm not saying that its not the PrintServer, which is in the users same domain, but I don't see anything out of the ordinary on the print server. I'm not even sure what print accounting is, but basically authenticated users are allowed to print on all the printers and CREATOR OWNER is allowed to manage jobs. The users have the ability to add (if they know how) any printer in the building that they want (with only a couple of exceptions). The 2 affected users don't share any common printers but they do share the only printserver we have. Just a thought is that that server PRINTSERV1 is also the WSUS server. Could the user's machine polling the WSUS server cause an account lockout?

So that GPO change did not work. 1 day without a lockout and then more lockouts today. I will remove the printers from the users computer for which should remove any connections (besides WSUS) with the PRINTSERV1.

Again, thanks for all the help with this.

I've given up on finding the cause. I had a friend mention that they'd seen the problem before and it had something to do with AD not liking the SID on the account anymore. So I've decided to delete the old user accounts, wait until Exchange sees them as orphaned, then create new user accounts with the same usernames and reassociate the Exchange mailbox with the newly created AD accounts. That will give the user the same login name and won't lose their email.

Thanks all for the help. I definitely know a lot more about the domain logon process than before. I gave the points to those who helped the most although I don't know that any definitive solution was provided.

Thanks.

Gee your welcome for my time