?
Solved

Looping a shell script

Posted on 2012-04-02
6
Medium Priority
?
462 Views
Last Modified: 2012-04-02
Hi,

I have a shell script (pipeline.sh) located in the following directory:

/fs/nas15/home/mqbpgsm4/permanalysis/bin

I would like to run 1000 instances of this script on a linux server by submitting it to a queue using Qsub 1000 times.

i.e:
qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh

I would like to loop it so that the job is submitted to server 1000 times:
i.e.

qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm2 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm3 sh pipeline.sh
to
qsub -b y (/directory/permanalysis/bin) -N perm1000 sh pipeline.sh

Is this at all possible?

Thanks,

Stephen
0
Comment
Question by:StephenMcGowan
6 Comments
 
LVL 1

Expert Comment

by:Xizz
ID: 37795308
Hi Stephen,

You could try something like this:

#!/bin/bash
for ((i=1;i<=1000;i++));
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
 
LVL 48

Expert Comment

by:Tintin
ID: 37795331
for i in $(seq 1 1000)
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
 
LVL 16

Accepted Solution

by:
Peter Kwan earned 2000 total points
ID: 37795365
Sure it is possible. The following is an example:

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y (/directory/permanalysis/bin) -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window

0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

 

Author Comment

by:StephenMcGowan
ID: 37795498
Hi Guys,

Thanks for getting back to me.

Would I need to define the bin folder for the location of pipeline.sh?

i.e:

#define pipeline.sh location
$directory=~/fs/nas15/home/mqbpgsm4/permanalysis/bin

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y $directory -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window


Thanks,

Stephen
0
 
LVL 1

Expert Comment

by:Xizz
ID: 37795570
Yes you could do that, otherwise place the script in the same folder.
0
 

Author Comment

by:StephenMcGowan
ID: 37795585
Sorry,

Just to double check, would the attached file work ok?
I just want to double check this out before potentially submitting 1000 jobs to the server.

Thanks again,

Stephen
loopscript.sh
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

589 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question