?
Solved

Service Unavailable - unusual Application Pool problem

Posted on 2010-01-13
10
Medium Priority
?
3,046 Views
Last Modified: 2012-08-14
One evening our web front end SharePoint server (MOSS on Win Srv 2003) spontaneously began spitting out W3SVC 1011 events. The day after our internal portal was unavailable. "Could not be found" first, and then "Service Unavailable", when the application pool is automatically shut down.

I have googled this literally for hours, and tried several solutions from MS Knowledgebase and others, but nothings helps.

I also have W3SVC warnings 1009, 1010, and 1013, and the problem is according to these related to the application pool. I have tried with different service accounts, seen to that the accounts have the rights and permissions it needs; created different application pools, and so forth and so on, but nothing makes any difference. - Other sites works.
0
Comment
Question by:Jack_A_Roe
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
10 Comments
 
LVL 13

Expert Comment

by:Springy555
ID: 26303203
Were there any windows updates made to the server recently?  Check this article:
http://support.microsoft.com/kb/2009746
0
 

Author Comment

by:Jack_A_Roe
ID: 26303339
Thanks for answering, but no, no updates were installed at the time, or for several days if not weeks prior to this. The update the KB article refers to is not installed.
0
 
LVL 13

Expert Comment

by:Springy555
ID: 26304019
Are these .net errors?  Can you paste the event log errors here?
0
Get proactive database performance tuning online

At Percona’s web store you can order full Percona Database Performance Audit in minutes. Find out the health of your database, and how to improve it. Pay online with a credit card. Improve your database performance now!

 

Author Comment

by:Jack_A_Roe
ID: 26304111
Here these W3SVC warnings are. I also have a recurring .NET error in the application log (se below); I can get rid of this by deleting files in the file system, but there's really no point, AFAIKS. Also, there are errors telling me that crawling can't begin because the content sources can't be found. Note that there's nothing wrong with the SQL server or its databases.

Central Administration works without problems. Test sites w. test application pools too. But if I put my portal site in the CA application pool, there's no difference.

Event Type:      Warning
Event Source:      W3SVC
Event Category:      None
Event ID:      1010
Date:            2010-01-13
Time:            14:32:42
User:            N/A
Computer:      MYSERVER
Description:
A process serving application pool 'My Application Pool' failed to respond to a ping. The process id was '5952'.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.


Event Type:      Warning
Event Source:      W3SVC
Event Category:      None
Event ID:      1011
Date:            2010-01-13
Time:            14:36:38
User:            N/A
Computer:      MYSERVER
Description:
A process serving application pool 'My Application Pool' suffered a fatal communication error with the World Wide Web Publishing Service. The process id was '636'. The data field contains the error number.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Data:
0000: 6d 00 07 80               m..€    


Event Type:      Error
Event Source:      .NET Runtime 2.0 Error Reporting
Event Category:      None
Event ID:      1000
Date:            2010-01-13
Time:            15:14:25
User:            N/A
Computer:      MYSERVER
Description:
Faulting application w3wp.exe, version 6.0.3790.3959, stamp 45d6968e, faulting module kernel32.dll, version 5.2.3790.4480, stamp 49c51f0a, debug? 0, fault address 0x0000bef7.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Data:
0000: 41 00 70 00 70 00 6c 00   A.p.p.l.
0008: 69 00 63 00 61 00 74 00   i.c.a.t.
0010: 69 00 6f 00 6e 00 20 00   i.o.n. .
0018: 46 00 61 00 69 00 6c 00   F.a.i.l.
0020: 75 00 72 00 65 00 20 00   u.r.e. .
0028: 20 00 77 00 33 00 77 00    .w.3.w.
0030: 70 00 2e 00 65 00 78 00   p...e.x.
0038: 65 00 20 00 36 00 2e 00   e. .6...
0040: 30 00 2e 00 33 00 37 00   0...3.7.
0048: 39 00 30 00 2e 00 33 00   9.0...3.
0050: 39 00 35 00 39 00 20 00   9.5.9. .
0058: 34 00 35 00 64 00 36 00   4.5.d.6.
0060: 39 00 36 00 38 00 65 00   9.6.8.e.
0068: 20 00 69 00 6e 00 20 00    .i.n. .
0070: 6b 00 65 00 72 00 6e 00   k.e.r.n.
0078: 65 00 6c 00 33 00 32 00   e.l.3.2.
0080: 2e 00 64 00 6c 00 6c 00   ..d.l.l.
0088: 20 00 35 00 2e 00 32 00    .5...2.
0090: 2e 00 33 00 37 00 39 00   ..3.7.9.
0098: 30 00 2e 00 34 00 34 00   0...4.4.
00a0: 38 00 30 00 20 00 34 00   8.0. .4.
00a8: 39 00 63 00 35 00 31 00   9.c.5.1.
00b0: 66 00 30 00 61 00 20 00   f.0.a. .
00b8: 66 00 44 00 65 00 62 00   f.D.e.b.
00c0: 75 00 67 00 20 00 30 00   u.g. .0.
00c8: 20 00 61 00 74 00 20 00    .a.t. .
00d0: 6f 00 66 00 66 00 73 00   o.f.f.s.
00d8: 65 00 74 00 20 00 30 00   e.t. .0.
00e0: 30 00 30 00 30 00 62 00   0.0.0.b.
00e8: 65 00 66 00 37 00 0d 00   e.f.7...
00f0: 0a 00                     ..      
0
 
LVL 15

Expert Comment

by:Tray896
ID: 26304598
Based on those errors you are going to need to do some deeper debugging.  Check out the debug diag tool.  It will generate a report when your process crashes that should hopefully point you towards a fix: http://support.microsoft.com/kb/919792
0
 
LVL 17

Expert Comment

by:Rovastar
ID: 26310285
Yeah I agree with Tray these look heavy.


Debug diag might be able to help but that is very heavy going I am not sure how man here will be able to help with that dump. Likely you will need MS support help with that unless it is something very obvious in teh dump.
0
 

Author Comment

by:Jack_A_Roe
ID: 26311769
Thanks for the replies. I've run the debag diag tool already, but it's pretty hard to interpret the outcome. There are several errors of the following type at the time of the crash (se below). And the crash---temination of processes---appears at couple of minutes after you access the site from a browser, at the same time a login box appears for the user.

In w3wp__PID__3612__Date__01_13_2010__Time_10_02_19AM__355__First chance exception 0XE06D7363.dmp the assembly instruction at kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft Corporation has caused an unknown exception (0xe06d7363) on thread 16

This exception originated from msvcr80!_CxxThrowException+46.

Review the faulting call stack for thread 16 to determine root cause for the exception.

Please follow up with vendor Microsoft Corporation for problem resolution concerning the following file: C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.3053_x-ww_B80FA8CA\msvcr80.dll.
Perhaps I need to try to rebuild this web application, but if it's bad code, it might follow me to the new one. I'm reluctant to call Microsoft Support, cause you usually end up talking to an Indian guy with an Irish accent, and I simply don't understand what they are saying.

0
 

Author Comment

by:Jack_A_Roe
ID: 26312196
One more thing, please note that while those dlls seem to be corrupt or something of the kind, every other web site works, for instance, Central Administration. If I put the portal site in the application pool in which CA is run, I get the same errors. If I put Central Administration in the application pool mentioned in the events above ("Portal Application Pool"), CA works. So the problem seems not to be the application pool, or the web service in it self, but this particular site. It is very strange because no one was working with the site when it decided to report the event, and no one had for several days (christmas holidays).
0
 

Author Comment

by:Jack_A_Roe
ID: 26316873
Update: While I can't browse to http://portal, I can browse to subsites, as in http://portal/sales, and I can browse to libraries on http://portal and read the document, if accessing them directly with apropriate URL:s. When I'm at a subsite, I can browse around as usual, however if I click "Portal", it is not accessible.

I installed a new SharePoint server, with a new farm, and a web application named Portal; removed the content db created in the process, and added a copy of the "portal db", which worked fine. However, the symptoms are exactly the same on the new server farm, as in the old.

So it certainly seems like it is the default.aspx page which is corrupt. I can access the master page and so forth in SharePoint Designer for instance, and I can create a new start page with the old hierarchy, fully browsable and so forth---but as soon as I go to the original start page, things hangs and I get the W3SCV-warning 1011 above, on the new and on the old farm.
0
 

Accepted Solution

by:
Jack_A_Roe earned 0 total points
ID: 26317319
So I could browse to a sub site, and from there I could get to the settings for the top page. There, I could reset to site definition. Now, my portal is accessible.
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When it comes to showing a 404 error page to your visitors, you do not want that generic page to show, and you especially do not want your hosting provider’s ad error page to show either. In this article, I will show you how to enable the custom 40…
When using a search centre, I'm going to show you how to configure Sharepoint's search to only return results from the current site collection. Very useful when using Office 365 with multiple site collections.
Visualize your data even better in Access queries. Given a date and a value, this lesson shows how to compare that value with the previous value, calculate the difference, and display a circle if the value is the same, an up triangle if it increased…
In this video, Percona Director of Solution Engineering Jon Tobin discusses the function and features of Percona Server for MongoDB. How Percona can help Percona can help you determine if Percona Server for MongoDB is the right solution for …

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question