[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

Looping a shell script

Posted on 2012-04-02
6
Medium Priority
?
461 Views
Last Modified: 2012-04-02
Hi,

I have a shell script (pipeline.sh) located in the following directory:

/fs/nas15/home/mqbpgsm4/permanalysis/bin

I would like to run 1000 instances of this script on a linux server by submitting it to a queue using Qsub 1000 times.

i.e:
qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh

I would like to loop it so that the job is submitted to server 1000 times:
i.e.

qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm2 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm3 sh pipeline.sh
to
qsub -b y (/directory/permanalysis/bin) -N perm1000 sh pipeline.sh

Is this at all possible?

Thanks,

Stephen
0
Comment
Question by:StephenMcGowan
6 Comments
 
LVL 1

Expert Comment

by:Xizz
ID: 37795308
Hi Stephen,

You could try something like this:

#!/bin/bash
for ((i=1;i<=1000;i++));
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
 
LVL 48

Expert Comment

by:Tintin
ID: 37795331
for i in $(seq 1 1000)
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
 
LVL 16

Accepted Solution

by:
Peter Kwan earned 2000 total points
ID: 37795365
Sure it is possible. The following is an example:

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y (/directory/permanalysis/bin) -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window

0
Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

 

Author Comment

by:StephenMcGowan
ID: 37795498
Hi Guys,

Thanks for getting back to me.

Would I need to define the bin folder for the location of pipeline.sh?

i.e:

#define pipeline.sh location
$directory=~/fs/nas15/home/mqbpgsm4/permanalysis/bin

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y $directory -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window


Thanks,

Stephen
0
 
LVL 1

Expert Comment

by:Xizz
ID: 37795570
Yes you could do that, otherwise place the script in the same folder.
0
 

Author Comment

by:StephenMcGowan
ID: 37795585
Sorry,

Just to double check, would the attached file work ok?
I just want to double check this out before potentially submitting 1000 jobs to the server.

Thanks again,

Stephen
loopscript.sh
0

Featured Post

Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Article by: Tammy
MySQLTuner is a script written in Perl that allows you to review a MySQL installation quickly and make adjustments to increase performance and stability. The current configuration variables and status data is retrieved and presented in a brief forma…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Six Sigma Control Plans
Suggested Courses

612 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question