Link to home
Start Free TrialLog in
Avatar of ccfcfc
ccfcfcFlag for United Kingdom of Great Britain and Northern Ireland

asked on

Windows service stopping - WINDOWS 2012 -

Running Windows 2012 with a service running under Window Services. For some reason, this at certain times stop and fails to restart even though service is set to Recovery to Restart. Looking at the Windows event log the following information has been written. BUt,  it makes no sense as to where I would be pointing the finger to as to the real issue i.e. Windows AD or there is a problem with the application or a GP problem.
See extract from event logs below :-


Fault bucket , type 0
Event Name: CLR20r3
Response: Not available
Cab Id: 0

Problem signature:
P1: intamacactivationhandlerservice
P2: 1.0.0.0
P3: 54f43942
P4: DAL
P5: 1.0.0.0
P6: 54f43938
P7: fd0
P8: 2a
P9: System.Data.StrongTyping
P10:

Attached files:

These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_intamacactivatio_6bc1839796871265e4a46367ca64c74af0a2ab_0325fadd

Analysis symbol:
Rechecking for solution: 0
Report Id: bbf0f759-cc1a-11e4-9449-0e071ace88f5
Report Status: 4
Hashed bucket:
Avatar of Cliff Galiher
Cliff Galiher
Flag of United States of America image

Well, that's .Net crashing. It could be a poorly coded service. Or it could be another app managed to replace some critical .Net dll's with incompatible versions. I've seen both variants.
Avatar of ccfcfc

ASKER

It gets a bit weirder. Every now and again, the .NET app which runs as a service stop and you have to re-enter its password and a dialogue box tells you it is re-assigning rights to run as a service. See extract from the event log. BUt, this is not every time.  So I re-enter the password in the service and all is fine. Nothing is locked out and the user account is set in the GP to be able to run as a batch/service.

From event log :-

An account failed to log on.

Subject:
      Security ID:            SYSTEM
      Account Name:            AMI01-APR-EON01$
      Account Domain:            AMI01
      Logon ID:            0x3E7

Logon Type:                  5

Account For Which Logon Failed:
      Security ID:            NULL SID
      Account Name:            srv_eonopenfire
      Account Domain:            AMI01

Failure Information:
      Failure Reason:            The user has not been granted the requested logon type at this machine.
      Status:                  0xC000015B
      Sub Status:            0x0

Process Information:
      Caller Process ID:      0x204
      Caller Process Name:      C:\Windows\System32\services.exe

Network Information:
      Workstation Name:      AMI01-APR-EON01
      Source Network Address:      -
      Source Port:            -

Detailed Authentication Information:
      Logon Process:            Advapi  
      Authentication Package:      Negotiate
      Transited Services:      -
      Package Name (NTLM only):      -
      Key Length:            0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
      - Transited services indicate which intermediate services have participated in this logon request.
      - Package name indicates which sub-protocol was used among the NTLM protocols.
      - Key length indicates the length of the generated session key. This will be 0 if no
Increasingly sounds like a code issue with the service/app itself. You'll likely need to work with the vendor/developer to solve the issue.
Avatar of ccfcfc

ASKER

Not entirely sure on that one as, why does when you try to restart the service windows reports you do not have the settings to run s a service. To resolve the password needs to re-entered and then it reports that the account is now able to run as a service. Surely thats an OS/AD issue ? Yet within the Group Policy this is how it is set already
Windows  credential management doesn't mean on .Net. But a .Net app, after starting, could easily change (or request to change) its security context and that can cause credentials to be expired. The OS is very likely behaving exactly as it is supposed to.
Avatar of ccfcfc

ASKER

So is that a setting in .Net or something in the .Net app ? We have probably 10 or more different .Net services and only a few of these exhibit this. The worrying or annoying part is, running a 24/7 environment, this can happen over night and needs the password re-entering.
Something in the .Net app, which is why I said you"\l need to work with the developer. If it were a setting or a corrupt OS, the issue would be more pervasive, or the OS itself would show significant signs of instability.
Avatar of ccfcfc

ASKER

From looking through the code, there is nothing that is explicitly doing anything with the credentials.
Whichever account the service is being run as is used for everything within the service application itself.
Well, I'm not going to try and convince you. I have better things to do with my time. I've stated what I believe the problem to be and have provided the appropriate evidence to back up that opinion and discount the OS or a widespread .Net issue. You can choose to accept that or not. You came here to ask for help, after all. Not the other way around.
Avatar of ccfcfc

ASKER

We are not dismissing your reply, but just asking for details on how you have come to this conclusion in previous situations. e.g. examples of what the code did to force this.
We would be grateful for evidence as many hours have been spent looking hence using this forum to pull upon experience of others.
So if you have details I would be ever so grateful othewise, I am no wiser and feel I am pointing the finger without any real evidence or experience which, in IT is a given.
ASKER CERTIFIED SOLUTION
Avatar of Cliff Galiher
Cliff Galiher
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of ccfcfc

ASKER

Well thanks for that, but trust me I have been through all that logic at the OS/AD level and with the developers.
I was hoping that you may have "real life" experience of perhaps a procedure or function within .Net that you have seen in the past.
I have spent many hours of "deductive" reasoning and before, I can confidently point the point at anything I need evidence of the issue hence by posting of this on this forum hoping someone comes forwrd with documental evidence of "real life" experience as, I also have done "deductive" steps.
I appreciate your time to reply and conents but you must understand, I have a team of developers whom will ask for evidence of issue which is what I am looking for.
Hah. And there's the catch-22.  I, in fact, said above that yes, the OS will react to certain events. I *know* this because of "real life experience."  I didn't pull that out of thin air. But if pointing to this thread won't convince you (or your developers) then why would my proclamation that I've seen it before suddenly change your mind??  It wouldn't!  So why should I bother?

Now, if you want me to review your code explicitly, c'mon...you know developers get paid good money.  As far as the rest, any developer worth their salt can add debug routines.  Take the whole credential thing out of it.  Your initial post was about a crash.  The credentials aren't playing a part in that. THAT you can take to your developers without speculation, without finger pointing, and say "here is a .Net crash" and they should be able to debug that. And if they don't...well...you have bigger problems than your services losing credentials.  

I've been doing DevOps before DevOps was even a term, so I wear the dual hat of developer and IT Pro more often than not. I know how things work and I know when it is an IT issue and when it is a developer issue. I'm not sure what you expected here, but I now believe you won't find it from *any* expert because you seem to be wanting more specifics than can be provided, given the environment. Unless you want to post your code and a full OS image of an existing machine with the issue so some expert who is more desparate than I, or who has WAY more time on their hands, can boot up the image, run checksums on every file, reproduce the bug, and/or scour code, I just don't know what you expected to happen here....
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of ccfcfc

ASKER

The resolution to the actual crash is with our Dev team to look at, but in the meantime we need to restart the service to get it back up and running again.

We have it configured to automatically restart on failure, but this isn't working.
When we try to restart the service manually, it requires us to re-enter the service user's password at which point it says that the user has been granted the logon as a service right (https://www.experts-exchange.com/questions/28638011/Windows-service-stopping-WINDOWS-2012.html?anchorAnswerId=40671186#a40671186)
Is there a specific reason that this service is bound to a user as opposed to Local Service, Network Service or Local System (which are the three standard Service accounts)?  When you introduce a user into the service mix, then you end up with the potential of an outside party exercising it's ability to potentially change the properties associated with said user (in other words, there could be an audit policy in place to prevent the user from having the LogonAs Service right).

-saige-
Avatar of ccfcfc

ASKER

The said user account is set within the Domain Policy to have the required permissions (logon as a service and batch) .
The reason specific user accounts are used is due to having a mulit-tenent platform with different services running different programs on a per customer basis. We can and need to control access to DB's etc.
The account WAS set to have the permissions to start as a service/batch already - domain policy based.
The fix was to re-type the service password and it restarted - as a dialogue box reported it was not set as this requirement.
This is a strange fix as the re-tying the password resolved the issue nothing was changed in the policy and looking at the GP this was still set as required. i,e LogonAs a Service  and Batch

Hope that makes sense
Does the user have *any* other policies applied to him/her (to be certain on this point, you can run an RSoP [Resultant Set of Policy] in logging mode to determine which policies affect the user)?

Is the user account actively used in any other process (meaning; for example, is the user used to login to a workstation and perform non-service related tasks or is the user used to authenticate other service related tasks)?

Is the user affected by the password auditing policy (assuming you have one)?

Is the server that this service is attached to a DC?  If not, have you attempted to create the user locally, assign them the appropriate rights and run the service?

-saige-
Avatar of ccfcfc

ASKER

The service account does not logon to a member server but is set to only run as a service on a member server.

How do I run thr RSoP against that service account ?
You will need to log into the server that the service runs on with the account in question, once a profile has been generated for the service account, you should then be able to run the RSoP in logging mode against the server for that service account.

-saige-
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial