Solved

fork() with ActivePerl 5.8.x on Win32 (Win2k) seems limited to approx 64 "children" - then exits - ideas/options?

Posted on 2003-11-27
6
986 Views
Last Modified: 2008-03-10
Hi all, here's the (basic) background and what I've done so far to test things:

I have been working on a "sleeper" process which occasionally wakes up and runs an executable (via exec()). I would rather not have the main loop of the "sleeper" process wait until the executable exits: instead I've used fork() to call a child process and exit that child once complete (the usual from what I've seen on the internet).

I know that the Win32 fork() emulation spawns a "fake" thread within the main perl executable's thread. This is okay for what I'm wanting to do.

Unfortunately the entire "sleeper" process seems to die after a time, no apparant reason why. I put together the following little codelet to test the fork() routine and see what might be happening:

----- code -----
#!c:/perl/bin/perl.exe

# use strict coding principles
use strict;

# local variable to count the loops
my $init = 0;

# do infinite loops (at least until a ctrl-c)
while (1) {
      # fork and put the "pid" into a local variable
      my $forked = fork();
      # check if we're in the child process
      if ($forked == 0) {
            # sleep for 0.01 of a second
            Win32::Sleep(10);
            # display the child's "pid" to be sure that it is 0 like it should be
            print "Forked - $forked\n";
            # exit the child process
            exit(0);
      }
      # increment the loop counter variable
      $init++;
      # display the "pid" of the just-created child and the loop
      print "PID: $forked - $init\n";
      # sleep for 0.1 of a second
      Win32::Sleep(100);
}
----- end code -----

Here is the results (roughly...):

----- result -----
C:\Perl\Projects\testing>test.pl
PID: -8736 - 1
Forked - 0
PID: -8856 - 2
Forked - 0
PID: -7832 - 3
Forked - 0
...
PID: -9004 - 63
Forked - 0
PID: -9008 - 64
Forked - 0
Forked -

C:\Perl\Projects\testing>
----- end result -----

My question(s) are the following: Is this due to my having implemented fork() improperly, or is this due to the way fork() has been implemented on Win32? Would it be better to use something like Win32::Thread instead of fork(), if it isn't that robust?

Thanks!
WilliamV
0
Comment
Question by:WilliamV
6 Comments
 
LVL 18

Assisted Solution

by:kandura
kandura earned 250 total points
Comment Utility
I tried your little script, and on Windows it stopped at 64 for me too. On linux it happily runs on until I ran out of patience ;^)
So it seems it's a limitation of the win32 fork emulation.

Why don't you use Win32::Process?  Seems a perfect match for what you're trying to do.
0
 
LVL 2

Expert Comment

by:ext2
Comment Utility
I confirmed the 64 fork limit here as well (ActivePerl 5.8.0/WinXP).  It also occurs on Perl 5.8.2/WinXP (self compiled).  However, Perl 5.8.0 under Cygwin (www.cygwin.com) under WinXP behaves like Linux and does not choke at 64 forks.  The likely reason is that the fork implementations are different: Perl under Cygwin can use Cygwin's emulation of fork under Win32, while native Perl must implement its own emulatation of fork on Win32.  perlwin32, http://aspn.activestate.com/ASPN/docs/ActivePerl/lib/Pod/perlwin32.html, mentions that fork emulation on native Perl was implemented by ActiveState at v. 5.6.

You might also try the platform-independent "threads" module for multithreading:

  http://search.cpan.org/~jhi/perl-5.8.1/ext/threads/threads.pm
  http://search.cpan.org/~jhi/perl-5.8.1/ext/threads/shared/shared.pm
0
 
LVL 24

Expert Comment

by:shivsa
Comment Utility
I tried on Solaris, and it runs fine....

u are right about fork on windows.

Windows where the fork() system call is not available, Perl can be built to emulate fork() at the interpreter level. While the emulation is designed to be as compatible as possible with the real fork() at the the level of the Perl program, there are certain important differences that stem from the fact that all the pseudo child ``processes'' created this way live in the same real process as far as the operating system is concerned.

please read this doc, it has all the limitation and all.
http://www.xav.com/perl/lib/Pod/perlfork.html

0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 2

Accepted Solution

by:
ext2 earned 250 total points
Comment Utility
In contrast, the following chokes on thread #249.  I'm not sure if this limit is system specific.  If you remove the sleep(100), however, in which case threads die, then it seems to go up indefinitely.

===========================
#!c:/perl/bin/perl.exe

# use strict coding principles
use strict;
use threads;

# local variable to count the loops
my $init = 0;

# do infinite loops (at least until a ctrl-c)
while (1) {
     # fork and put the "pid" into a local variable
     my $forked = async {
          # sleep for 0.01 of a second
          sleep(0.01); #Win32::Sleep(10);
          # display the child's "pid" to be sure that it is 0 like it should be
          print "Forked\n";
          sleep(100);
     }
     # increment the loop counter variable
     $init++;
     # display the "pid" of the just-created child and the loop
     print "PID: $forked - $init\n";
     # sleep for 0.1 of a second
     #Win32::Sleep(100);
     sleep(0.01);
}
===========================
0
 

Author Comment

by:WilliamV
Comment Utility
Wow! Thanks for the options guys...I'll have a look at them and check to see which will work better for the situation I'm using (I'm leaning towards the Win32::Process, thank you kandura). I'll probably increase the points and split them between kandura and ext2, within the next day or so.

Thanks very much! :)
WilliamV
0
 

Author Comment

by:WilliamV
Comment Utility
Hi everyone, sorry that it took so long to do the split on. My testing took longer than I expected (drat work!). I also couldn't figure how to add points to the question (thought that was an option but guess I was wrong).

Cheers!
WilliamV
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This video demonstrates how to create an example email signature rule for a department in a company using CodeTwo Exchange Rules. The signature will be inserted beneath users' latest emails in conversations and will be displayed in users' Sent Items…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now