Multithreading in Bash. Possible?

I am trying to get a concept where I can create simultaneous running child processes from a parent and able to read their exit status as soon as they come in. Once the exit code is read it will start a new child to take its place. That way there is alway 10 children running at once.

In the end I am going to be using this for a way to do asynchronous wgets to speed up my wget scripting. So if you have an alternative, let me know. Thats also why I set a randomized timer on the example child process to simulate wgets and so I can check if it doesnt do things in order or not.

Here is the child process I am threading, just for reference:

[CODE]number=0
let "number = $RANDOM % 20"
time=$(echo "1+$number*0.1" | bc)
sleep $time
echo "I got $1 ($time)"
exit $1
[/CODE]


I first started out with this job script I found. I realized that it didnt get the exit codes fast enough. I wanted the exit code retrieved as soon as the process is exited.

[CODE]

#!/bin/bash

MAXJOBS=10

function jobidfromstring()
{
        local STRING;
        local RET;
 
        STRING=$1;
        RET="$(echo $STRING | sed 's/^[^0-9]*//' | sed 's/[^0-9].*$//')"
 
        echo $RET;
}
 
function clearToSpawn
{
    local JOBCOUNT="$(jobs -r | grep -c .)"
    if [ $JOBCOUNT -lt $MAXJOBS ] ; then
        echo 1;
        return 1;
    fi
 
    echo 0;
    return 0;
}
 
JOBLIST=""
 
for (( i=0; i<=20; i++ ));
do
        while [ `clearToSpawn` -ne 1 ] ; do
                sleep 0.5
        done  
        ./child $i &
        LASTJOB=`jobidfromstring $(jobs %%)`
       JOBLIST="$JOBLIST $LASTJOB"
done

for JOB in $JOBLIST ; do
        wait %$JOB
        echo "Job $JOB exited with status $?"
done
echo "Done."[/CODE]


Then I decided to tryout the SIGCHLD handler and see how that worked. It turned out that I wasn't able to define the handler in a way that would capture every child for some reason.

[CODE]#!/bin/bash

iNext=0

function ChildReturned() {
        wait $1
        echo "$1 was returned. Its exits status was: $?"
        iNext=$((iNext+1))
        StartChild $iNext
}


function StartChild {
        ./child $1 &
        pid=$!
        echo "Started: $1  PID= $pid"
        trap 'ChildReturned $pid' SIGCHLD
}


set -bm

for (( iNext=0; iNext<=10; iNext++ ));
do
        StartChild $iNext
done

wait
[/CODE]


Finally I tried just starting the children and looping through a while statement that would check when each one died, retrieve the exit code, and then start a new child. However I realized that a child that terminates (SIGCHLD) still exists as a process until you get the exit code with wait(). Therefore using kill -0 wasnt going to work.

[CODE]#!/bin/bash


throttle=10
iNext=0
JobIDArray[0]=""

function jobidfromstring()
{
        local STRING;
        local RET;

        STRING=$1;
        RET="$(echo $STRING | sed 's/^[^0-9]*//' | sed 's/[^0-9].*$//')"
        echo $RET;
}

function AnyAlive {
        for (( i=1; i<=throttle; i++ )); do
                if kill -0 %${JobIDArray[i]} > /dev/null 2>&1; then
                        echo 1
                        return 1
                fi
        done
        echo 0
        return 0
}

for (( iNext=1; iNext<=throttle; iNext++ )); do
        ./child $iNext &
        JobIDArray[iNext]=`jobidfromstring $(jobs %%)`
        echo "          JobIDArray[$iNext] = ${JobIDArray[iNext]}"
done


while [[ `AnyAlive` -eq 1 ]] ; do
        for (( i=1; i<=throttle; i++ )); do
                jID=${JobIDArray[i]}
                if ! kill -0 %$jID; then
                        wait %$jID
                        return=$?
                        iNext=$((iNext+1))
                        ./child $iNext &
                        JobIDArray[i]=`jobidfromstring $(jobs %%)`
                        echo "$jID was returned. Its exits status was: $return   |   JobIDArray[$i] = ${JobIDArray[i]}"
                fi
        done
done[/CODE]



---------------------------

I am pretty much stuck at where to begin next. I am very new to bash and I don't even know if its possible to do what I want. My third script would work fine if I just had a way to identify if a job was terminated or not. I know that if you try to kill -s SIGCHLD the job when its terminated, it will give "no job exists" or something. I can't seem to know how I can get that to work in an IF statement. And if I did, would wait() still get the exit code.Start Free Trial