Solved

New child processes failed with 0xc06d007e error after they are launched

Posted on 2015-02-18
4
541 Views
Last Modified: 2015-07-19
Hello, I have some server process on Win2008 Server R2 64bit OS which starts many subprocesses for clients and
sometimes new child processes start to fail with 0xc06d007e exitcode right after their are launched in the moment while loading DLL-s. Such process didn't reach the main() function from main exe module. It's not caused by lower level of desktop heap, because we increase it and 90% desktop heap is free in the critical time.
When start of child processes begins to fail, all child processes fail on the same error. And even simple console application from the another developers could not starts and exits with the same error (this console application runs in the same windows session as the main server app). When a number of running child processes is decreased, error disappear and everything works fine. I suppose there is some system resource exhausted but I have no idea which is it.
Main server and child processes are 32bit applications. Main server app has about 200-500 child processes running at the same time. It could be about 10000 - 20000 processes started and finished for about 5 hours when this situation occurs.
Child processes communicates with main server process with pipes (each child process uses 4 pipes and 2 synchronization events). Because of this child processes inherits handles from main server process.
Does somebody have idea what should be wrong?

Thanks

Petr Penas

Here is the snapshot of system when the problem occurred:

Server session

  Desktop Heap Size:    3145728 (0x  300000) Bytes
  Committed:              380928  (0x   5d000) Bytes
  UnCommitted:            2764800 (0x  2a3000) Bytes
  Allocated:              367848  (0x   59ce8) Bytes
  Total Freed              13080   (0x    3318) Bytes
  Unused:                  2777880 (0x  2a6318) Bytes
  Used Rate:                       (        11)%

*** Virtual Memory Usage ***
    Physical Memory:    15728510 (  62914040 Kb)

************ NO PAGING FILE *********************

Unimplemented error for MiSystemVaTypeCount
    Available Pages:     5466841 (  21867364 Kb)
    ResAvail Pages:     15227274 (  60909096 Kb)
    Locked IO Pages:           0 (         0 Kb)
    Free System PTEs:   33554639 ( 134218556 Kb)
    Modified Pages:         5181 (     20724 Kb)
    Modified PF Pages:      5176 (     20704 Kb)
    NonPagedPool 0 Used:       0 (         0 Kb)
    NonPagedPoolNx 0 Used: 67246 (    268984 Kb)
    NonPagedPool 1 Used:       0 (         0 Kb)
    NonPagedPoolNx 1 Used: 22889 (     91556 Kb)
    NonPagedPool 2 Used:       0 (         0 Kb)
    NonPagedPoolNx 2 Used: 17254 (     69016 Kb)
    NonPagedPool 3 Used:       0 (         0 Kb)
    NonPagedPoolNx 3 Used: 22112 (     88448 Kb)
    NonPagedPool Usage: 33666032 ( 134664128 Kb)
    NonPagedPoolNx Usage: 143368 (    573472 Kb)
    NonPagedPool Max:   11757539 (  47030156 Kb)
    ********** Excessive NonPaged Pool Usage *****
    PagedPool 0 Usage:    126752 (    507008 Kb)
    PagedPool 1 Usage:     30288 (    121152 Kb)
    PagedPool 2 Usage:      4856 (     19424 Kb)
    PagedPool 3 Usage:      6881 (     27524 Kb)
    PagedPool 4 Usage:      7918 (     31672 Kb)
    PagedPool Usage:      176695 (    706780 Kb)
    PagedPool Maximum:  33554432 ( 134217728 Kb)
    Shared Commit:         24610 (     98440 Kb)
    Special Pool:              0 (         0 Kb)
    Shared Process:        77535 (    310140 Kb)
    PagedPool Commit:     176841 (    707364 Kb)
    Driver Commit:          3772 (     15088 Kb)
    Committed pages:    10298563 (  41194252 Kb)
    Commit limit:       15728044 (  62912176 Kb)
0
Comment
Question by:literaigor
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
4 Comments
 
LVL 20

Expert Comment

by:compdigit44
ID: 40617831
Is the process 32 or 64bit? Is the process Java or .Net based etc....

have you tried running Process Explorer or Process Monitor
0
 

Author Comment

by:literaigor
ID: 40618414
As I said, main server and child processes are 32bit applications. It use C++ runtime and yes, I've tried process explorer, but didn't seen any system resource on low level...
0
 

Accepted Solution

by:
literaigor earned 0 total points
ID: 40880860
The problem was finally solved. It was caused by a third-party component (LibXL.dll) loaded into the starting processes. This component in some cases did not released GDI handles. When the number of GDI handles in Windows session reached technical maximum (65536), other processes started to fail at their start in the initialization method gdi32.dll!_NtGdiInit@0 (). Fortunately, newer version of this component already has the leak of GDI objects resolved.

The problem was also discussed in more detail in MSDN forums in the thread "New child processes failed with 0xc0000142 (or 0xc06d007e) error after they are launched".
0
 

Author Closing Comment

by:literaigor
ID: 40888239
After a lot of debugging and system state snapshots we have found the core of the problem.

Our parent service is running under the SYSTEM account, and when the user (administrator) tried to view the number of GDI handles of its process, he saw 0, probably due to insufficient permissions. Only running the service in the current session of the interactive user (our architecture allows that for debugging purposes) allows to see the actual number of GDI handles used by the service as well as its subprocesses. Thousands of unreleased GDI handles among those processes that were used to call the aforementioned third-party library (LibXL.dll), then finally helped identify the core of the problem.
0

Featured Post

Office 365 Training for Admins

Learn how to provision tenants, synchronize on-premise Active Directory, and implement Single Sign-On with these master level course.  Only from Platform Scholar

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I had a question today where the user wanted to know how to delete an SSL Certificate, so I thought that I would quickly add this How to! Article for your reference. WHY WOULD YOU WANT TO DELETE A CERTIFICATE? 1. If an incorrect certificate was …
A procedure for exporting installed hotfix details of remote computers using powershell
This tutorial will walk an individual through the steps necessary to install and configure the Windows Server Backup Utility. Directly connect an external storage device such as a USB drive, or CD\DVD burner: If the device is a USB drive, ensure i…
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…

740 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question