Link to home
Start Free TrialLog in
Avatar of Member_2_6492660_1
Member_2_6492660_1Flag for United States of America

asked on

Sharepoint 2010 Application Pools will not remain Started

SharePoint 2010
Windows 212 R2
SQL  2014

Last night I had several power outages in my home.  I run a some network here systems are on UPS but they had enough

After power up my Web Server Widow 2012 R2 all looked good.  I have Nagios that monitors my Application pools

The following three application pools will not stay started.

SecurityTokenApplicationPool

Sharepoint- 8000

SharePoint Default App Pool

I have restarted all the SharePoint services
I restarted IIS  

So my SharePoint is down now

Short of jus restarting the server any idea?

Thank you

Tom
Avatar of Piotr Strycharz
Piotr Strycharz
Flag of Poland image

If pools are in stopped state for some reason (manual or some service action) they will remain stopped forever despite restarting IIS or server. For me it happens sometimes (rarely) when upgrading packages (maybe the actual reason is different). You must start them manually. Use IIS manager to do it. If they will not start - the reason is provided in the EventLog.
Check the event viewer logs and the SharePoint logs, using a ULS Viewer.

Also check the user account to make sure it is not locked or a password was changed?
Avatar of Member_2_6492660_1

ASKER

Guys thanks for the replies.

The accounts are not locked and the passwords are set to never expire and have not been changed they are service accounts.


Getting event id 2159  8306  5555 1 and 4 repeated

Log Name:      Application
Source:        Microsoft-SharePoint Products-SharePoint Foundation
Date:          11/14/2018 21:36:22
Event ID:      2159
Task Category: Unified Logging Service
Level:         Error
Keywords:      
User:          OUR\spfarm
Computer:      TGCS013.our.network.tgcsnet.com
Description:
Event 8306 (SharePoint Foundation) of severity 'Error' occurred 36 more time(s) and was suppressed in the event log
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-SharePoint Products-SharePoint Foundation" Guid="{6FB7E0CD-52E7-47DD-997A-241563931FC2}" />
    <EventID>2159</EventID>
    <Version>14</Version>
    <Level>2</Level>
    <Task>88</Task>
    <Opcode>0</Opcode>
    <Keywords>0x4000000000000000</Keywords>
    <TimeCreated SystemTime="2018-11-15T02:36:22.799234500Z" />
    <EventRecordID>569675</EventRecordID>
    <Correlation />
    <Execution ProcessID="7028" ThreadID="14896" />
    <Channel>Application</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security UserID="S-1-5-21-3054588571-1341459584-784128302-3142" />
  </System>
  <EventData>
    <Data Name="int0">8306</Data>
    <Data Name="string1">SharePoint Foundation</Data>
    <Data Name="int2">36</Data>
  </EventData>
</Event>

Log Name:      Application
Source:        Microsoft-SharePoint Products-SharePoint Portal Server
Date:          11/14/2018 21:01:02
Event ID:      5555
Task Category: User Profiles
Level:         Critical
Keywords:      
User:          OUR\spfarm
Computer:      TGCS013.our.network.tgcsnet.com
Description:
Failure trying to synch web application 9689b150-b5cd-4c69-99a8-62ffe60a5aa7, ContentDB fbb3a22b-676d-4636-b80c-eccc3d16455a  Exception message was Microsoft.Office.Server.UserProfiles.UserProfileApplicationNotAvailableException: No User Profile Application available to service the request. Contact your farm administrator.
   at Microsoft.Office.Server.Administration.UserProfileApplicationProxy.get_ApplicationProperties()
   at Microsoft.Office.Server.Administration.UserProfileApplicationProxy.get_PerfmonInstanceHandle()
   at Microsoft.Office.Server.UserProfiles.ContentDBSynchronizer..ctor(ELogType logType, SPContentDatabase cdb, SPJobState jobState)
   at Microsoft.Office.Server.UserProfiles.WSSProfileSyncJob.Execute()
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-SharePoint Products-SharePoint Portal Server" Guid="{8B3DDD3D-2B09-4669-BF81-E2D6921FEEEA}" />
    <EventID>5555</EventID>
    <Version>14</Version>
    <Level>1</Level>
    <Task>8</Task>
    <Opcode>0</Opcode>
    <Keywords>0x4000000000000000</Keywords>
    <TimeCreated SystemTime="2018-11-15T02:01:02.316447800Z" />
    <EventRecordID>569659</EventRecordID>
    <Correlation ActivityID="{0452F3C0-6597-4A2B-A964-490981EEE540}" />
    <Execution ProcessID="7028" ThreadID="16180" />
    <Channel>Application</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security UserID="S-1-5-21-3054588571-1341459584-784128302-3142" />
  </System>
  <EventData>
    <Data Name="string0">Failure trying to synch web application 9689b150-b5cd-4c69-99a8-62ffe60a5aa7, ContentDB fbb3a22b-676d-4636-b80c-eccc3d16455a</Data>
    <Data Name="string1">Microsoft.Office.Server.UserProfiles.UserProfileApplicationNotAvailableException: No User Profile Application available to service the request. Contact your farm administrator.
   at Microsoft.Office.Server.Administration.UserProfileApplicationProxy.get_ApplicationProperties()
   at Microsoft.Office.Server.Administration.UserProfileApplicationProxy.get_PerfmonInstanceHandle()
   at Microsoft.Office.Server.UserProfiles.ContentDBSynchronizer..ctor(ELogType logType, SPContentDatabase cdb, SPJobState jobState)
   at Microsoft.Office.Server.UserProfiles.WSSProfileSyncJob.Execute()</Data>
  </EventData>
</Event>

Log Name:      Application
Source:        Microsoft-SharePoint Products-SharePoint Foundation
Date:          11/13/2018 20:39:00
Event ID:      8306
Task Category: Claims Authentication
Level:         Error
Keywords:      
User:          OUR\spfarm
Computer:      TGCS013.our.network.tgcsnet.com
Description:
An exception occurred when trying to issue security token: The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc/actas is too busy. .
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-SharePoint Products-SharePoint Foundation" Guid="{6FB7E0CD-52E7-47DD-997A-241563931FC2}" />
    <EventID>8306</EventID>
    <Version>14</Version>
    <Level>2</Level>
    <Task>47</Task>
    <Opcode>0</Opcode>
    <Keywords>0x4000000000000000</Keywords>
    <TimeCreated SystemTime="2018-11-14T01:39:00.944286300Z" />
    <EventRecordID>568935</EventRecordID>
    <Correlation ActivityID="{D57C81AB-7745-4E3B-A869-7451E517FDD7}" />
    <Execution ProcessID="13464" ThreadID="15344" />
    <Channel>Application</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security UserID="S-1-5-21-3054588571-1341459584-784128302-3142" />
  </System>
  <EventData>
    <Data Name="string0">The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc/actas is too busy. </Data>
  </EventData>
</Event>

Log Name:      Application
Source:        Microsoft-SharePoint Products-PerformancePoint Service
Date:          11/13/2018 01:42:43
Event ID:      1
Task Category: PerformancePoint Services
Level:         Error
Keywords:      
User:          OUR\spfarm
Computer:      TGCS013.our.network.tgcsnet.com
Description:
Failed to execute SharePoint BI Maintenance Job

Microsoft.PerformancePoint.Scorecards.BpmException: An unexpected error occurred.  Error 52068.  Additional details have been logged for your administrator.
   at Microsoft.PerformancePoint.Scorecards.BIMonitoringServiceApplicationProxy.ExecuteOnChannel(CodeBlock codeBlock)
   at Microsoft.PerformancePoint.Scorecards.BIMaintenanceJob.Execute(Guid targetInstanceId)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-SharePoint Products-PerformancePoint Service" Guid="{A7CD5295-CBBA-4DCA-8B67-D5BE061B6FAE}" />
    <EventID>1</EventID>
    <Version>14</Version>
    <Level>2</Level>
    <Task>1</Task>
    <Opcode>0</Opcode>
    <Keywords>0x4000000000000000</Keywords>
    <TimeCreated SystemTime="2018-11-13T06:42:43.878015200Z" />
    <EventRecordID>568462</EventRecordID>
    <Correlation ActivityID="{FC654DE5-6A0E-4DA2-A3F1-8F820EFC97A7}" />
    <Execution ProcessID="516" ThreadID="2504" />
    <Channel>Application</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security UserID="S-1-5-21-3054588571-1341459584-784128302-3142" />
  </System>
  <EventData>
    <Data Name="string1">Failed to execute SharePoint BI Maintenance Job

Microsoft.PerformancePoint.Scorecards.BpmException: An unexpected error occurred.  Error 52068.  Additional details have been logged for your administrator.
   at Microsoft.PerformancePoint.Scorecards.BIMonitoringServiceApplicationProxy.ExecuteOnChannel(CodeBlock codeBlock)
   at Microsoft.PerformancePoint.Scorecards.BIMaintenanceJob.Execute(Guid targetInstanceId)</Data>
  </EventData>
</Event>


Log Name:      Application
Source:        Microsoft-SharePoint Products-PerformancePoint Service
Date:          11/13/2018 01:42:43
Event ID:      4
Task Category: PerformancePoint Services
Level:         Error
Keywords:      
User:          OUR\spfarm
Computer:      TGCS013.our.network.tgcsnet.com
Description:
An unexpected error occurred.  Error 52068.

Exception details:
System.ServiceModel.ServerTooBusyException: The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc/actas is too busy.  ---> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Server stack trace:
   at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message)
   at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse& rstr)
   at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)
   at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo)
   at Microsoft.SharePoint.SPSecurityContext.<>c__DisplayClass7.<GetProcessSecurityTokenForServiceContext>b__6()
   at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)
   at Microsoft.SharePoint.SPSecurityContext.GetProcessSecurityTokenForServiceContext()
   at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForServiceContext(Uri contextUri)
   at Microsoft.SharePoint.SPChannelFactoryOperations.InternalCreateChannelActingAsLoggedOnUser[TChannel](ChannelFactory`1 factory, EndpointAddress address, Uri via)
   at Microsoft.SharePoint.SPChannelFactoryOperations.CreateChannelActingAsLoggedOnUser[TChannel](ChannelFactory`1 factory, EndpointAddress address)
   at Microsoft.PerformancePoint.Scorecards.BIMonitoringServiceApplicationProxy.GetChannel(Uri address)
   at Microsoft.PerformancePoint.Scorecards.BIMonitoringServiceApplicationProxy.ExecuteOnChannel(CodeBlock codeBlock)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-SharePoint Products-PerformancePoint Service" Guid="{A7CD5295-CBBA-4DCA-8B67-D5BE061B6FAE}" />
    <EventID>4</EventID>
    <Version>14</Version>
    <Level>2</Level>
    <Task>1</Task>
    <Opcode>0</Opcode>
    <Keywords>0x4000000000000000</Keywords>
    <TimeCreated SystemTime="2018-11-13T06:42:43.862388600Z" />
    <EventRecordID>568461</EventRecordID>
    <Correlation ActivityID="{FC654DE5-6A0E-4DA2-A3F1-8F820EFC97A7}" />
    <Execution ProcessID="516" ThreadID="2504" />
    <Channel>Application</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security UserID="S-1-5-21-3054588571-1341459584-784128302-3142" />
  </System>
  <EventData>
    <Data Name="int1">52068</Data>
    <Data Name="string2">System.ServiceModel.ServerTooBusyException: The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc/actas is too busy.  ---&gt; System.Net.WebException: The remote server returned an error: (503) Server Unavailable.
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Server stack trace:
   at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&amp; msgData, Int32 type)
   at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message)
   at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse&amp; rstr)
   at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)
   at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo)
   at Microsoft.SharePoint.SPSecurityContext.&lt;&gt;c__DisplayClass7.&lt;GetProcessSecurityTokenForServiceContext&gt;b__6()
   at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)
   at Microsoft.SharePoint.SPSecurityContext.GetProcessSecurityTokenForServiceContext()
   at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForServiceContext(Uri contextUri)
   at Microsoft.SharePoint.SPChannelFactoryOperations.InternalCreateChannelActingAsLoggedOnUser[TChannel](ChannelFactory`1 factory, EndpointAddress address, Uri via)
   at Microsoft.SharePoint.SPChannelFactoryOperations.CreateChannelActingAsLoggedOnUser[TChannel](ChannelFactory`1 factory, EndpointAddress address)
   at Microsoft.PerformancePoint.Scorecards.BIMonitoringServiceApplicationProxy.GetChannel(Uri address)
   at Microsoft.PerformancePoint.Scorecards.BIMonitoringServiceApplicationProxy.ExecuteOnChannel(CodeBlock codeBlock)</Data>
  </EventData>
</Event>

I have new updates to apply and a pending reboot for sure

I restarted all service and IIS reset no change.
ASKER CERTIFIED SOLUTION
Avatar of Piotr Strycharz
Piotr Strycharz
Flag of Poland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks guys I figured it out you pointed me to the correct place

 The event cam up

Log Name:      System
Source:        Microsoft-Windows-WAS
Date:          11/16/2018 10:02:00
Event ID:      5021
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      TGCS013.our.network.tgcsnet.com
Description:
The identity of application pool SecurityTokenServiceApplicationPool is invalid. The user name or password that is specified for the identity may be incorrect, or the user may not have batch logon rights. If the identity is not corrected, the application pool will be disabled when the application pool receives its first request.  If batch logon rights are causing the problem, the identity in the IIS configuration store must be changed after rights have been granted before Windows Process Activation Service (WAS) can retry the logon. If the identity remains invalid after the first request for the application pool is processed, the application pool will be disabled. The data field contains the error number.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
    <EventID Qualifiers="32768">5021</EventID>
    <Version>0</Version>
    <Level>3</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2018-11-16T15:02:00.000000000Z" />
    <EventRecordID>618373</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>System</Channel>
    <Computer>TGCS013.our.network.tgcsnet.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="AppPoolID">SecurityTokenServiceApplicationPool</Data>
    <Binary>69050780</Binary>
  </EventData>
</Event>


I cleared all the events after I restarted the server for updates today.

I started the apppool  and received the above message

Hear is what happened I created a new gpo to use restricted groups and in that gpo I placed two accounts in the logon as a batch policy

I had to add the two accounts that I was using for the app pools now they are working