Solved

Job scheduling in Unix

Posted on 2015-01-05
9
124 Views
Last Modified: 2015-01-19
Hi,

I have 10 shell scripts which need to be run in a sequence manner. In my early works i used Autosys but now we cant afford it .

Is there any free schedulers available so i can configure it ? Ie. script1.sh is parent and script2.sh will execute only after script1.sh completed and script3.sh will execute only after completion of script1.sh and script2.sh etc.

Please advice.

Thanks,
magento
0
Comment
Question by:magento
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 40

Assisted Solution

by:omarfarid
omarfarid earned 150 total points
ID: 40532908
you can use crontab to schedule jobs on unix / linux systems.

For more info on using crontab, please see:

http://code.tutsplus.com/tutorials/scheduling-tasks-with-cron-jobs--net-8800

Also, you can have one master shell script scheduled with crontab to run at specific times, and that master script will run your scripts in sequence, e.g. master script could be like below:

/path/to/script1.sh
/path/to/script2.sh
/path/to/script3.sh
/path/to/script4.sh
/path/to/script5.sh
/path/to/script6.sh
/path/to/script7.sh
/path/to/script8.sh
/path/to/script9.sh
/path/to/script10.sh

The above will run the scripts in sequence and the start of next script will be after the completion of the previous one.

Please note while scheduling you need to take care of overlapping between crontab jobs. e.g. if running the scripts takes more than an hour, then you need to run the job every 1 hour, then you need to take care of the overlap of jobs
0
 
LVL 5

Author Comment

by:magento
ID: 40532924
I will check the timing, but in worst case if it takes more time , Is it a good idea i can use lock for the master script and so the cron wont start until the lock of PID for master script completed?
0
 
LVL 5

Author Comment

by:magento
ID: 40533051
Also what is script5.sh was failed ? so i think this not going to help my situation?
0
A new era in Cloud training has arrived.

A day that will go down in Cloud history.. But are you ready for it? Will you accept this Cloud challenge?

 
LVL 40

Expert Comment

by:omarfarid
ID: 40533083
At the beginning of master script you can check if there is a lock and accordingly proceed.

At the end of the master script you need to remove the lock. Be careful here where the lock will be there if the master script exit before removing the lock.

For the case  where a script is failing in between, you can check the the exit status of the previous script by checking the value of $? , if it is 0 (zero) then the scrip exit successfully other value means error, and accordingly proceed with other scripts run.
0
 
LVL 21

Expert Comment

by:tfewster
ID: 40534225
Regarding Free scheduling software, a web search for "linux job scheduling software freeware" shows many possibilities, and even a Wikipedia comparison of the better known ones.  If your scheduling is more complex than "run scripts in sequence on a single server (and halt if one fails)" and if you're familiar with AutoSys you'll know what functionality you need.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 40534478
Do you want the any of the scripts to run in parallel?

What behaviour do you want if any of the scripts fail?

How long will they usually take to run?

How often do you want to schedule them?
0
 
LVL 5

Author Comment

by:magento
ID: 40534563
Hi Tintin,

Do you want the any of the scripts to run in parallel?
>>I want them to run in an sequence manner.
What behaviour do you want if any of the scripts fail?
>>If script4.sh failed, then it should be stopped right there.
How long will they usually take to run?
>>This will finish within 20 mins
How often do you want to schedule them?
>>Every minute.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 350 total points
ID: 40534616
Schedule a master script in cron, that has:

#!/bin/bash
# Exit if another copy of this script is running
pidof -s -o '%PPID' $(basename $0) >/dev/null 2>&1 && exit 1

function run {
  eval $*

  if [ $? -ne 0 ]
  then
        echo "$* failed"
        exit 2
  fi
}

for i in {1..10}
do
   run /path/to/script$i
done

Open in new window

0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40534839
If running the scripts takes 20 min, how do you expect to schedule to run every min? There will be overlap
0

Featured Post

Automating Terraform w Jenkins & AWS CodeCommit

How to configure Jenkins and CodeCommit to allow users to easily create and destroy infrastructure using Terraform code.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
In part one, we reviewed the prerequisites required for installing SQL Server vNext. In this part we will explore how to install Microsoft's SQL Server on Ubuntu 16.04.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
Suggested Courses

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question