Solved

New child processes failed with 0xc06d007e error after they are launched

Posted on 2015-02-18
4
512 Views
Last Modified: 2015-07-19
Hello, I have some server process on Win2008 Server R2 64bit OS which starts many subprocesses for clients and
sometimes new child processes start to fail with 0xc06d007e exitcode right after their are launched in the moment while loading DLL-s. Such process didn't reach the main() function from main exe module. It's not caused by lower level of desktop heap, because we increase it and 90% desktop heap is free in the critical time.
When start of child processes begins to fail, all child processes fail on the same error. And even simple console application from the another developers could not starts and exits with the same error (this console application runs in the same windows session as the main server app). When a number of running child processes is decreased, error disappear and everything works fine. I suppose there is some system resource exhausted but I have no idea which is it.
Main server and child processes are 32bit applications. Main server app has about 200-500 child processes running at the same time. It could be about 10000 - 20000 processes started and finished for about 5 hours when this situation occurs.
Child processes communicates with main server process with pipes (each child process uses 4 pipes and 2 synchronization events). Because of this child processes inherits handles from main server process.
Does somebody have idea what should be wrong?

Thanks

Petr Penas

Here is the snapshot of system when the problem occurred:

Server session

  Desktop Heap Size:    3145728 (0x  300000) Bytes
  Committed:              380928  (0x   5d000) Bytes
  UnCommitted:            2764800 (0x  2a3000) Bytes
  Allocated:              367848  (0x   59ce8) Bytes
  Total Freed              13080   (0x    3318) Bytes
  Unused:                  2777880 (0x  2a6318) Bytes
  Used Rate:                       (        11)%

*** Virtual Memory Usage ***
    Physical Memory:    15728510 (  62914040 Kb)

************ NO PAGING FILE *********************

Unimplemented error for MiSystemVaTypeCount
    Available Pages:     5466841 (  21867364 Kb)
    ResAvail Pages:     15227274 (  60909096 Kb)
    Locked IO Pages:           0 (         0 Kb)
    Free System PTEs:   33554639 ( 134218556 Kb)
    Modified Pages:         5181 (     20724 Kb)
    Modified PF Pages:      5176 (     20704 Kb)
    NonPagedPool 0 Used:       0 (         0 Kb)
    NonPagedPoolNx 0 Used: 67246 (    268984 Kb)
    NonPagedPool 1 Used:       0 (         0 Kb)
    NonPagedPoolNx 1 Used: 22889 (     91556 Kb)
    NonPagedPool 2 Used:       0 (         0 Kb)
    NonPagedPoolNx 2 Used: 17254 (     69016 Kb)
    NonPagedPool 3 Used:       0 (         0 Kb)
    NonPagedPoolNx 3 Used: 22112 (     88448 Kb)
    NonPagedPool Usage: 33666032 ( 134664128 Kb)
    NonPagedPoolNx Usage: 143368 (    573472 Kb)
    NonPagedPool Max:   11757539 (  47030156 Kb)
    ********** Excessive NonPaged Pool Usage *****
    PagedPool 0 Usage:    126752 (    507008 Kb)
    PagedPool 1 Usage:     30288 (    121152 Kb)
    PagedPool 2 Usage:      4856 (     19424 Kb)
    PagedPool 3 Usage:      6881 (     27524 Kb)
    PagedPool 4 Usage:      7918 (     31672 Kb)
    PagedPool Usage:      176695 (    706780 Kb)
    PagedPool Maximum:  33554432 ( 134217728 Kb)
    Shared Commit:         24610 (     98440 Kb)
    Special Pool:              0 (         0 Kb)
    Shared Process:        77535 (    310140 Kb)
    PagedPool Commit:     176841 (    707364 Kb)
    Driver Commit:          3772 (     15088 Kb)
    Committed pages:    10298563 (  41194252 Kb)
    Commit limit:       15728044 (  62912176 Kb)
0
Comment
Question by:literaigor
  • 3
4 Comments
 
LVL 19

Expert Comment

by:compdigit44
ID: 40617831
Is the process 32 or 64bit? Is the process Java or .Net based etc....

have you tried running Process Explorer or Process Monitor
0
 

Author Comment

by:literaigor
ID: 40618414
As I said, main server and child processes are 32bit applications. It use C++ runtime and yes, I've tried process explorer, but didn't seen any system resource on low level...
0
 

Accepted Solution

by:
literaigor earned 0 total points
ID: 40880860
The problem was finally solved. It was caused by a third-party component (LibXL.dll) loaded into the starting processes. This component in some cases did not released GDI handles. When the number of GDI handles in Windows session reached technical maximum (65536), other processes started to fail at their start in the initialization method gdi32.dll!_NtGdiInit@0 (). Fortunately, newer version of this component already has the leak of GDI objects resolved.

The problem was also discussed in more detail in MSDN forums in the thread "New child processes failed with 0xc0000142 (or 0xc06d007e) error after they are launched".
0
 

Author Closing Comment

by:literaigor
ID: 40888239
After a lot of debugging and system state snapshots we have found the core of the problem.

Our parent service is running under the SYSTEM account, and when the user (administrator) tried to view the number of GDI handles of its process, he saw 0, probably due to insufficient permissions. Only running the service in the current session of the interactive user (our architecture allows that for debugging purposes) allows to see the actual number of GDI handles used by the service as well as its subprocesses. Thousands of unreleased GDI handles among those processes that were used to call the aforementioned third-party library (LibXL.dll), then finally helped identify the core of the problem.
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Remote Apps is a feature in server 2008 which allows users to run applications off Remote Desktop Servers without having to log into them to run the applications.  The user can either have a desktop shortcut installed or go through the web portal to…
OfficeMate Freezes on login or does not load after login credentials are input.
This tutorial will walk an individual through locating and launching the BEUtility application and how to execute it on the appropriate database. Log onto the server running the Backup Exec database. In a larger environment, this would generally be …
This tutorial will walk an individual through the steps necessary to join and promote the first Windows Server 2012 domain controller into an Active Directory environment running on Windows Server 2008. Determine the location of the FSMO roles by lo…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now