Type 3 logon failures 4625 from old credentials

Fred Marshall
Fred Marshall used Ask the Experts™
on
We have a couple of workstations (of many) on a domain which are generating Windows logon errors 4625 Type 3 at a workstation "fileserver".  I've not been able to find a cause - so they persist.
Our SIEM system is reporting them.  
I've made traffic captures with wireshark between the client and the server but don't see anything very helpful.
The usernames (such as would be cached in Windows credentials) that are failing are no longer in use and hark back to a time when these workstations were on a peer-to-peer network and Wikndows Credentials *were* in use.

I saw: https://www.experts-exchange.com/questions/29168030/Failed-Logons-hammering-Workgroup-file-server-Win-10.html
The thing is, the possible usernames are now domain users and have replaced the local usernames that had been in use earlier.  
We have removed all instances of Windows Credentials that had been in use during domain transition.
(The fileserver workstation is in the last subnet to be brought into the domain).

There are lots of examples where this isn't a problem.   There are quite a few workstation fileservers and over 50 workstations.  The one fileserver affected is the most accessible by number of users.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Distinguished Expert 2017

Commented:
Are you talking about credentials from non domain based credentials?
Type 3 as you noted is a network access attemp. On the fileserver sevurity log if enabled by audit GPO could shed light on the source of the request.
It could be a saved credential with old info.

Are accesses to the share manual by the user, or through GPO?

Have the old credentials been disabled, I.e. Users can no longer use their non domain credentials to login?

Try to narrow the issue to the source. Scanning these events in the file server to determine which workstation/s they are coming from.

There are powershell cmdlets that can scan through the sevurity event log extracting the data that could help you narrow down the source/s.
Paul MacDonaldDirector, Information Systems

Commented:
I'd look for older mapped drives, as well as startup scripts/batch files that attempt to map drives programmatically.  You can find persistent maps in File Explorer or in the registry.  Don't ignore Group Policy scripts that run on logon, even though it's counter-intuitive.

After that, I would look for any apps - especially older ones - which may have been configured to read or save data to the file server using old credentials.

Starting a computer in safe mode, then having the user log in may provide some clues.
Distinguished Expert 2017

Commented:
I Agree with Paul's comment, but as the author noted, there are 50 workstations so narrowing down the issue to the workstation that are generating this.

The following might be helpful to start with:
https://devblogs.microsoft.com/scripting/use-powershell-cmdlet-to-filter-event-log-for-easy-parsing/

Author

Commented:
arnold:
Are you talking about credentials from non domain based credentials?
Yes
Type 3 as you noted is a network access attemp. On the fileserver sevurity log if enabled by audit GPO could shed light on the source of the request.
As I said, we get this information through our SIEMl.  It sheds light BUT while it identifies the client COMPUTER, it doesn't identify the Windows logon (if any).
It could be a saved credential with old info.
Yes it could.  That's why we worked to delete them all from all user profiles.

Are accesses to the share manual by the user, or through GPO?
Permissions are provided via group membership.

Have the old credentials been disabled, I.e. Users can no longer use their non domain credentials to login?
As above, Windows Credentials have been deleted from the cache.

Try to narrow the issue to the source. Scanning these events in the file server to determine which workstation/s they are coming from.
As in the first part of the question.  This *is* narrowed down.

There are powershell cmdlets that can scan through the sevurity event log extracting the data that could help you narrow down the source/s.
As above, we have the sources.  USERNAME (used for access), REMOTE DEVICE (which is always the client computername) and DOMAIN (which is sometimes the client computername but is usually the domain name).
Distinguished Expert 2017

Commented:
Where are you looking for these
The account name should be part of the event
https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4625

Did the workstations have scheduled tasks, or services that were setup to run as local users?


The profiles of the non domain users were they migrated to be domain user accounts? or the domain users started with new profiles and data copied in?

What cache? Do the local users still exist in the workstation's ?

Author

Commented:
Paul McDonald:  Thank you!
I'd look for older mapped drives, as well as startup scripts/batch files that attempt to map drives programmatically.
Have done this already.  
One that you didn't mention and one that I've also investigated is this:
If a fileserver is listed in  "Quick access", it appears to behave as a MAP as well.
We've not used startup scripts or batch files for MAPs.  In fact, I've been discouraging MAPping.

You can find persistent maps in File Explorer or in the registry.  Don't ignore Group Policy scripts that run on logon, even though it's counter-intuitive.
There are no persistent maps.
There are no GPOs that wouldn't apply more widely.

After that, I would look for any apps - especially older ones - which may have been configured to read or save data to the file server using old credentials
. I found quite a few entries in the Registry of this sort that referred to the server IP address and paths.  But, I didn't see where there were any credentials associated with them.  So how the usernames being used are invoked remains a mystery.

Starting a computer in safe mode, then having the user log in may provide some clues.
There doesn't seem to be "the user" identified.  There can be quite a few local profiles and it seems this happens with no logon at all.
Distinguished Expert 2017

Commented:
Scan the system for malware, originating these requests.
Distinguished Expert 2017

Commented:
You may have events when people open the network where all current systems are enumerated.
Distinguished Expert 2017

Commented:
test, login as a local user and see whether hitting windows+E and then navigate and expand network whether you'll get this event.

Author

Commented:
arnold:  I understand but that's a bit of an open-ended quest.  It's a bit hard to imagine it happening when the offending workstations aren't logged on.  But, I guess Scheduled Tasks might?  Also a bit hard to imagine this one thing (expanding the network) would be done on schedule.  Also, these things aren't happening on schedule....  
Any other insights that might help us trim this down before investigating?
Distinguished Expert 2017

Commented:
First thing on the server where the shares are, to identify the workstations from which request attempts are coming.

If it comes from all, do you have anti-virus apps installed running under local system and scan network resources in addition to local ones?

Author

Commented:
arnold:  OK.  Well maybe this will help.  It's a copy of a single failure report:
An account failed to log on.
Subject: Security ID: S-1-0-0
Account Name: -
Account Domain: -
Logon ID: 0x0
Logon Type: 3 <<<<<I already knew this
Account For Which Logon Failed: Security ID: S-1-0-0
Account Name: staff <<<<<I already knew this
Account Domain: scanner <<<<<I already knew this but this is the server-end machine name and NOT the domain name
Failure Information: Failure Reason: Account currently disabled. Status: 0xC000006E
Sub Status: 0xC0000072
Process Information:
Caller Process ID: 0x0
Caller Process Name: -
Network Information:
Workstation Name: 1LB <<<<<I already knew this
Source Network Address: 10.0.0.152 <<<<<I already knew this
Source Port: 53349
Detailed Authentication Information:
Logon Process: NtLmSsp
Authentication Package:
NTLM Transited Services: -
Package Name (NTLM only): -
Key Length: 0
This event is generated when a logon request fails. It is generated on the computer where access was attempted. The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe. The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network). The Process Information fields indicate which account and process on the system requested the logon. The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases. The authentication information fields provide detailed information about this specific logon request. - Transited services indicate which intermediate services have participated in this logon request. - Package name indicates which sub-protocol was used among the NTLM protocols. - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

So, I'm still trying to find out what's causing this.  Any insights here that I missed?
Distinguished Expert 2017

Commented:
Local account staff@scanner has an agent on each workstation through which it initiates scans?

Author

Commented:
No.  There can't be a local account "@scanner" except on "scanner".  And, I know of no agent on "scanner".  
I believe the report says this:
Account name: staff (a disabled account on both computers).
Computer: 1LB at 10.0.0.152
Tries to access files on "scanner"
Which fails.

So, one might ask, what is causing an attempt from "1LB" to access "scanner" using "staff"
Distinguished Expert 2017

Commented:
Do you have agents Installed on workstations through which it monitors the systems or manages?

Based on the log you posted, the server saw a request from a workstation that ran under the local credentials.

Author

Commented:
Yes.  That's what it's doing.  The question is why?
Once more, there is no agent on this workstation.
There are agents on other workstations but none that tie these two computers together.
The "server" in this case is just a file server workstation.  
The "client" in this case would just be a fileserver client.
Distinguished Expert 2017

Commented:
Here is the dilemma.
If scanner\staff only rubs on scanner, does scanner have a static or dynamic ip?
If scanner has a dynamic ip, the records reflected may indicate the change in ip, while local DNS has potentiall multiple, or not-up to date records.

Is this a different environment from your other question, domain joined?

What does scanner's task?

In a peer to peer, do all workstations have staff with the same password?
What is the frequency of these event log entries?

Author

Commented:
All of the computers have static IP addresses.
The event log entries happen at a frequency determined by the failures.
There is no peer-to-peer - although there had been.

This is probably too much information because it's all historical:
"scanner" is a fileserver workstation and one of the last to join the domain.
"1LB" is a client and one of the first to join the domain.
So, "1LB" had credentials for "staff" to match "staff" on "scanner" prior to "scanner" joining the domain - and of course they had the same password.
And "scanner" files had sharing permissions for "staff".
Now, users on "1LB" are members of a group with access to shared "scanner" files and there are no Windows credentials being used any longer.

Which other question, specifically?

Scanner's task, as before, is a fileserving workstation.

If scanner\staff only rubs on scanner
???? What does this mean?  Oh!  Maybe "runs", eh?  Well, "staff" no longer exists in the sense that it is NOT an Active User on any computer here.
Distinguished Expert 2017

Commented:
here is my thought:
Scanner\staff historically was it used to nap /access the share?
Adding file share auditing to see if it  can see what specific file, directory is being attempted or ..

The setup for the prior peer to peer system, might have that was setup to map a drive using the common credential.

Look at schtasks /query, look at services, look at local group policy.
Control keymgr.dll to see if a user has credentials to the share ...... Saved as scanner\staff or .\staff.....

Author

Commented:
Earlier, I said:
We have removed all instances of Windows Credentials that had been in use during domain transition.
(This also means BEFORE domain transition).

Since this is happening on only 2 computers, it seems unlikely that we've missed anything so obvious.
schtasks /query reveals nothing - not too unexpected because we didn't have any scheduled tasks for this kind of access.
etc.

We need a better diagnostic procedure...
Distinguished Expert 2017

Commented:
Check services. Check tasklist /fi "username eq staff"

The two systems in question, what did their role was in the peer to peer setup?

The transition "peer to peer" to AD transition .....

Author

Commented:
Here's an hypothesis that I'm testing now:
(I still don't understand why more client workstations don't cause this failed logon to happen)
(Our practice had been to set up desktop shortcuts rather than MAPs because MAPs use up a connection and shortcuts don't.
Since the "server" is a workstation, it's limited to 20 connections - so conservation is prudent in a situation where occasionally hitting the limit occurs).
Anyway, the offending clients have desktop shortcuts to the "server" named "scanner" or its IP address equivalent.
Because of the difference between a shortcut and a MAP (and I've already mentioned that a "Quick access" entry acts like a MAP re: connections), I had assumed that credentials would not be used for a shortcut UNTIL it was used.
This hypothesis suggests that's not the case and that there may be some "refresh" happening associated with the shortcuts.  
AND, the mysterious refresh carries along with it, the old credentials.
In many cases, the shortcuts were copied from peer-to-peer user desktops into domain user desktops under the assumption that they were simply "pointers" and nothing else.
So, under this hypothesis, the shortcuts are the offending element.
I've removed them and await today's report to see if it matters.
Yet, I suspect that there are quite a few more workstations with this condition that aren't causing the problem....

This raises a collateral question about shortcuts to network folders and security settings.  
I'll post it separately.
Distinguished Expert 2017

Commented:
I do not believe "shortcuts" on the desktop are the issue.
The shortcut does not retain credentials of all it has is \\ipaddress\share or \\scanner\share.

Do these two workstation include an auto-login when started?
There is sonething common to those that have /cause ......
An application or something that runs under preset credentials

Author

Commented:
No autologons.
Lots of things common to these and other workstations - but none worthy of note that I can tell.
Who *knows* exactly about apps?  But none that would need this particular access that I can see.
Distinguished Expert 2017
Commented:
something is a miss here, all workstations only provide local or ad domain logins, no microsoft logins that are mapped to the local staff account, correct?

this is the only remaining thing that I can think of that there is a microsoft logon that is tied to a local staff account and when this user logs in it tries to resume sessions to the shares.

Do you have login policy that records when a user logs in?
In this case, it would be a local user and the login recording process has to run from the local group policy.
it might be worth while settings up on the two offending workstations the login policy.
using mmc, under file add/remove group policy object, and under the user configuration, windows settings, scripts
login you can have a simple login script
@ECHO OFF
echo on, %DATE% at %TIME%, User: %USERNAME% loged in on %COMPUTERNAME% via  %CONSOLE% authorized by %LOGONSERVER% >> [to a local file]

make sure the file is writeable by all and the path where it is is accessible..
in a local access, you can not place it in a share, so after a few days following this setup. you would need to check. you could access remotely if ........

Author

Commented:
Thanks all!

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial