Looping a shell script

Hi,

I have a shell script (pipeline.sh) located in the following directory:

/fs/nas15/home/mqbpgsm4/permanalysis/bin

I would like to run 1000 instances of this script on a linux server by submitting it to a queue using Qsub 1000 times.

i.e:
qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh

I would like to loop it so that the job is submitted to server 1000 times:
i.e.

qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm2 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm3 sh pipeline.sh
to
qsub -b y (/directory/permanalysis/bin) -N perm1000 sh pipeline.sh

Is this at all possible?

Thanks,

Stephen
StephenMcGowanAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

XizzCommented:
Hi Stephen,

You could try something like this:

#!/bin/bash
for ((i=1;i<=1000;i++));
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
TintinCommented:
for i in $(seq 1 1000)
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
Peter KwanAnalyst ProgrammerCommented:
Sure it is possible. The following is an example:

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y (/directory/permanalysis/bin) -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

StephenMcGowanAuthor Commented:
Hi Guys,

Thanks for getting back to me.

Would I need to define the bin folder for the location of pipeline.sh?

i.e:

#define pipeline.sh location
$directory=~/fs/nas15/home/mqbpgsm4/permanalysis/bin

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y $directory -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window


Thanks,

Stephen
0
XizzCommented:
Yes you could do that, otherwise place the script in the same folder.
0
StephenMcGowanAuthor Commented:
Sorry,

Just to double check, would the attached file work ok?
I just want to double check this out before potentially submitting 1000 jobs to the server.

Thanks again,

Stephen
loopscript.sh
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Perl

From novice to tech pro — start learning today.