• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 332
  • Last Modified:

exec background process, capture output, and return value in an anonymous array

what i wish to do is really simple. I don't know how complex the solution is in perl. being the pismerl misaster that you are, i figured i'd ask you.

i need a function. the function is passed 2 arguments.  1st a string, which is a program/script name. 2nd a string containing the program/script's arguments. The function should be non-blocking, and promptly return($reference) while executing the (program + arguments) in the background also capturing the output and return value of the process.

upon completion of execution the return/output values should be set in hash reference as follows:

$reference->{PID}->{return} = "return value";
$reference->{PID}->{output} = "program output";


0
tejinashi
Asked:
tejinashi
  • 11
  • 9
  • 2
1 Solution
 
Adam314Commented:
When the function first returns, there will be no output and no return value from the program.  Those wouldn't be available until the program was finished executing.  Is this what you want?
0
 
tejinashiAuthor Commented:
Yes, I guess the function should return the PID right away, and update an external hash when the process completes
0
 
jhurstCommented:
sub yourFunction {
  my $pid=fork();
   if ($pid) {return $pid}
   my $command="$_[0] $_[1]";
    exec($command);
    }
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
tejinashiAuthor Commented:
so since this is actually forking would the only way to get the output of the command being execed back to the program who called the function be via IPC or sockets of some sort?
0
 
Adam314Commented:
I was thinking using threads, and placing the data in a shared variable.  If you are using any non thread safe modules though, this won't work.
0
 
tejinashiAuthor Commented:
I am worried about using threaded perl because I have heard of many issues with it. I am comfortable compiling perl with thread support. Can you tell me more about modules which aren't thread safe? Basically at this time I am using no external perl modules. Unless you call "strict" a module ;).

0
 
Adam314Commented:
If you aren't using any modules, then using threads will be okay.  Some modules, in their documentation, will state whether they are thread safe.  

Using threads, getting the data from the process running the external command to the main process will be easy (simply place it in a shared variable).
Using fork, getting the data back to the main process will be more difficult.  But if you are concerned about threads based on other things, using fork might be the "safest" way.
0
 
tejinashiAuthor Commented:
Using fork what is the best way to get the return output of the executed programs back to the main program? (I will raise the point value of this topic)

Cheers
0
 
jhurstCommented:
if you use the fork then you will need to have the forked copy send its output to a file, someything like $command="$_[0] $_[1] >/tmp/outputFromCommand.$$"

Now, the output will go to named file.  The one thing that I suspect that we are all missing here is why do you want to do this in the background if you want the script to wait for it to happen.  If you want to wait, then why not just do it.  If you can not wait then cleraly the outut will not be available.
0
 
tejinashiAuthor Commented:
Sorry I didn't explain that, the reason is because I could be executing as many as 30 to 40 different processes per second.
0
 
Adam314Commented:
There are several methods to get the output back to the main process.  See here for your options:
http://perldoc.perl.org/perlipc.html

Whatever method, you'll have to be sure to put the output into the right variable (eg: don't put child 1's output in the variable for child 2).  Once you have the method to get the data, this shouldn't be too difficult, you could have the child send the PID before the actual output.

Again, this is why I was considering threads.  The output could be placed in a shared variable.  The main program wouldn't have to do anything to get the data, it would simply be in the shared variable.
0
 
tejinashiAuthor Commented:
Using threads that same shared variable could be a hash or any other data structure that perl provides correct?

I will attempt to compile a threaded perl.
0
 
Adam314Commented:
Yes, it should work.
0
 
tejinashiAuthor Commented:
I have just received word that this should work without a thread enabled perl. After some research I can see that based on your recommendation that a threaded implementation should be much easier to work with.

I believe that in this case, we should provide a solution for both threaded and non-threaded implementations of perl. I have tripled the point value. 125 for the original question using a threaded implementation, and 250 for the non-threaded example. Does this sound about right?

I have succesfully compiled a threaded version of perl using:
sh Configure -Dprefix=/usr/local/wansecurity -Dusethreads -de

Make test ran fine.
0
 
Adam314Commented:
The solutions will be very different.  With the threaded implementation, you would put all of the output data into a shared variable - and it will be available in the main thread.

With the non-threaded implementation, you will have to use one of the IPC methods to get the data back into your main program.  As long as you have to do all of the development work to make it work with the non-threaded, there is no need to make it work with threads (as the non-threaded method will work on a threaded compiled perl).
0
 
tejinashiAuthor Commented:
I guess the best bet is to go with the non-threaded route.

I was just thinking though for the bennifit of the list, that it would be nice to post a solution for both methods. Hence the reason that I raised the point value.
0
 
tejinashiAuthor Commented:
Well, how about the threaded version then?
0
 
Adam314Commented:
Here is the threaded version.


#!/usr/bin/perl
use strict;
use threads;
use threads::shared;
use Data::Dumper;

my %reference: shared;

for(my $i=0; $i<10; $i++){
      my $newtid=do_command("program","$i");
      print "TID: $newtid\n";
}
print Dumper(\%reference);
#If it is possible for the main program to finish before all it's threads are finished, they should be joined first

sub do_command {
      my $thread;
      unless($thread=threads->create(\&do_command_thread,@_)){
            print STDERR "ERROR creating thread: $!\n";
      }
      return $thread->tid;
}

sub do_command_thread {
      my $program=shift;
      my $args=join(" ",@_);
      
      #To actually call the external program
      my @lines=`$program $args`;
      my $retvalue=$?;
      
      #For debugging, don't actually call external program
      #my @lines=("Running $program $args\n", "line1\n", "line2\n", "line3\n");
      #my $retvalue=10;
      
      my $tid=threads->self->tid;
      $reference{$tid}=&share({});
      $reference{$tid}->{'return'}=$retvalue;
      $reference{$tid}->{'complete'}=1;
      $reference{$tid}->{'output'}="@lines";
}

0
 
tejinashiAuthor Commented:
Thank you, my son was just born, so I need just a little bit of time to review this
0
 
Adam314Commented:
congratulations!

no problems, take as much time as needed.
0
 
tejinashiAuthor Commented:
I got a chance to test this out, and it works with a threaded perl to get return output back to the main process. Is there any way we can get an IPC version?
0
 
Adam314Commented:
I don't have much experience with IPC.

If you want to try it, the first thing you should do is read up on it:
http://perldoc.perl.org/perlipc.html

Then decide which method would work best for what you need (maybe post another question to get help with that)
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

  • 11
  • 9
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now