fork() with ActivePerl 5.8.x on Win32 (Win2k) seems limited to approx 64 "children" - then exits - ideas/options?

Hi all, here's the (basic) background and what I've done so far to test things:

I have been working on a "sleeper" process which occasionally wakes up and runs an executable (via exec()). I would rather not have the main loop of the "sleeper" process wait until the executable exits: instead I've used fork() to call a child process and exit that child once complete (the usual from what I've seen on the internet).

I know that the Win32 fork() emulation spawns a "fake" thread within the main perl executable's thread. This is okay for what I'm wanting to do.

Unfortunately the entire "sleeper" process seems to die after a time, no apparant reason why. I put together the following little codelet to test the fork() routine and see what might be happening:

----- code -----
#!c:/perl/bin/perl.exe

# use strict coding principles
use strict;

# local variable to count the loops
my $init = 0;

# do infinite loops (at least until a ctrl-c)
while (1) {
      # fork and put the "pid" into a local variable
      my $forked = fork();
      # check if we're in the child process
      if ($forked == 0) {
            # sleep for 0.01 of a second
            Win32::Sleep(10);
            # display the child's "pid" to be sure that it is 0 like it should be
            print "Forked - $forked\n";
            # exit the child process
            exit(0);
      }
      # increment the loop counter variable
      $init++;
      # display the "pid" of the just-created child and the loop
      print "PID: $forked - $init\n";
      # sleep for 0.1 of a second
      Win32::Sleep(100);
}
----- end code -----

Here is the results (roughly...):

----- result -----
C:\Perl\Projects\testing>test.pl
PID: -8736 - 1
Forked - 0
PID: -8856 - 2
Forked - 0
PID: -7832 - 3
Forked - 0
...
PID: -9004 - 63
Forked - 0
PID: -9008 - 64
Forked - 0
Forked -

C:\Perl\Projects\testing>
----- end result -----

My question(s) are the following: Is this due to my having implemented fork() improperly, or is this due to the way fork() has been implemented on Win32? Would it be better to use something like Win32::Thread instead of fork(), if it isn't that robust?

Thanks!
WilliamV
WilliamVAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

kanduraCommented:
I tried your little script, and on Windows it stopped at 64 for me too. On linux it happily runs on until I ran out of patience ;^)
So it seems it's a limitation of the win32 fork emulation.

Why don't you use Win32::Process?  Seems a perfect match for what you're trying to do.
0
ext2Commented:
I confirmed the 64 fork limit here as well (ActivePerl 5.8.0/WinXP).  It also occurs on Perl 5.8.2/WinXP (self compiled).  However, Perl 5.8.0 under Cygwin (www.cygwin.com) under WinXP behaves like Linux and does not choke at 64 forks.  The likely reason is that the fork implementations are different: Perl under Cygwin can use Cygwin's emulation of fork under Win32, while native Perl must implement its own emulatation of fork on Win32.  perlwin32, http://aspn.activestate.com/ASPN/docs/ActivePerl/lib/Pod/perlwin32.html, mentions that fork emulation on native Perl was implemented by ActiveState at v. 5.6.

You might also try the platform-independent "threads" module for multithreading:

  http://search.cpan.org/~jhi/perl-5.8.1/ext/threads/threads.pm
  http://search.cpan.org/~jhi/perl-5.8.1/ext/threads/shared/shared.pm
0
shivsaCommented:
I tried on Solaris, and it runs fine....

u are right about fork on windows.

Windows where the fork() system call is not available, Perl can be built to emulate fork() at the interpreter level. While the emulation is designed to be as compatible as possible with the real fork() at the the level of the Perl program, there are certain important differences that stem from the fact that all the pseudo child ``processes'' created this way live in the same real process as far as the operating system is concerned.

please read this doc, it has all the limitation and all.
http://www.xav.com/perl/lib/Pod/perlfork.html

0
OWASP Proactive Controls

Learn the most important control and control categories that every architect and developer should include in their projects.

ext2Commented:
In contrast, the following chokes on thread #249.  I'm not sure if this limit is system specific.  If you remove the sleep(100), however, in which case threads die, then it seems to go up indefinitely.

===========================
#!c:/perl/bin/perl.exe

# use strict coding principles
use strict;
use threads;

# local variable to count the loops
my $init = 0;

# do infinite loops (at least until a ctrl-c)
while (1) {
     # fork and put the "pid" into a local variable
     my $forked = async {
          # sleep for 0.01 of a second
          sleep(0.01); #Win32::Sleep(10);
          # display the child's "pid" to be sure that it is 0 like it should be
          print "Forked\n";
          sleep(100);
     }
     # increment the loop counter variable
     $init++;
     # display the "pid" of the just-created child and the loop
     print "PID: $forked - $init\n";
     # sleep for 0.1 of a second
     #Win32::Sleep(100);
     sleep(0.01);
}
===========================
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
WilliamVAuthor Commented:
Wow! Thanks for the options guys...I'll have a look at them and check to see which will work better for the situation I'm using (I'm leaning towards the Win32::Process, thank you kandura). I'll probably increase the points and split them between kandura and ext2, within the next day or so.

Thanks very much! :)
WilliamV
0
WilliamVAuthor Commented:
Hi everyone, sorry that it took so long to do the split on. My testing took longer than I expected (drat work!). I also couldn't figure how to add points to the question (thought that was an option but guess I was wrong).

Cheers!
WilliamV
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Perl

From novice to tech pro — start learning today.