Solved

Looping a shell script

Posted on 2012-04-02
6
452 Views
Last Modified: 2012-04-02
Hi,

I have a shell script (pipeline.sh) located in the following directory:

/fs/nas15/home/mqbpgsm4/permanalysis/bin

I would like to run 1000 instances of this script on a linux server by submitting it to a queue using Qsub 1000 times.

i.e:
qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh

I would like to loop it so that the job is submitted to server 1000 times:
i.e.

qsub -b y (/directory/permanalysis/bin) -N perm1 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm2 sh pipeline.sh
qsub -b y (/directory/permanalysis/bin) -N perm3 sh pipeline.sh
to
qsub -b y (/directory/permanalysis/bin) -N perm1000 sh pipeline.sh

Is this at all possible?

Thanks,

Stephen
0
Comment
Question by:StephenMcGowan
6 Comments
 
LVL 1

Expert Comment

by:Xizz
ID: 37795308
Hi Stephen,

You could try something like this:

#!/bin/bash
for ((i=1;i<=1000;i++));
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
 
LVL 48

Expert Comment

by:Tintin
ID: 37795331
for i in $(seq 1 1000)
do
   qsub -b y (/directory/permanalysis/bin) -N $i sh pipeline.sh
done

Open in new window

0
 
LVL 16

Accepted Solution

by:
Peter Kwan earned 500 total points
ID: 37795365
Sure it is possible. The following is an example:

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y (/directory/permanalysis/bin) -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window

0
VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

 

Author Comment

by:StephenMcGowan
ID: 37795498
Hi Guys,

Thanks for getting back to me.

Would I need to define the bin folder for the location of pipeline.sh?

i.e:

#define pipeline.sh location
$directory=~/fs/nas15/home/mqbpgsm4/permanalysis/bin

X=1
while [ $X -le 1000 ];
do
    eval "qsub -b y $directory -N perm$X sh pipeline.sh"
    X=`expr $X + 1`
done

Open in new window


Thanks,

Stephen
0
 
LVL 1

Expert Comment

by:Xizz
ID: 37795570
Yes you could do that, otherwise place the script in the same folder.
0
 

Author Comment

by:StephenMcGowan
ID: 37795585
Sorry,

Just to double check, would the attached file work ok?
I just want to double check this out before potentially submitting 1000 jobs to the server.

Thanks again,

Stephen
loopscript.sh
0

Featured Post

Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
AWS - HAProxy- KeepAlived 5 43
Linux script delete files 3 34
Impact of disabling SMB v1 on Mac and Linux clients 4 377
Create an automated page index 9 16
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question