Shell script to create Flash Archives (flar) on Solaris spawns hundreds of processes

Hello All

I need to write a shell script to create a flar image of our systems - This is the backup solution of 2 systems on an isolated LAN and has been dictated by budget and other technical constraints...

I find that my script will work fine if I run it myself from the command line, however, if I run it under cron I get hundreds of flar processes and the system gets itself in a mess!

The scripts is very simple, as follows:

cd /export/home/backups/local
rm *
flar create -n flar /export/home/backups/local/backup.flar
exit 1

That's it!

I think that I may have fundamentally missed something when running scripts under cron - Can anyone see anything amiss with this?

I can't really understand why something would work when instigated manually, but not work when run under cron...

Many thanks in advance for any help

Marcus
marcusjonesAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Hanno P.S.IT Consultant and Infrastructure ArchitectCommented:
a) which user is this running under (in which user's crontab did you put the command)?
b) Do you redirect the command's output to some file (logfile)?
    If not, did you check the user's email if you got some message?
c) If email doesn't work reliably, redirect your script's output into a file. Something like
        /my/script.sh > /var/log/flarcreate.log 2>&1
    should do.
d) Make sure the command "flar" can be found by using /usr/sbin/flar instead of flar
    alone.
0
Brian UtterbackPrinciple Software EngineerCommented:
How often does the cron job run? Are the hundreds of flar processes coming from different invocations via cron,
or are they all coming from a single run? Try using the ptree command to find out how the processes were
created. Try to use truss and pstack to find out what they are doing or waiting for.
0
marcusjonesAuthor Commented:
Thanks for your responses

The job is set to run under root's cron and /usr/sbin/ is in the PATH for all users, but maybe it would be neater to have it specified using the abosolute path in the script...

I'll try the troubelshooting suggestions that you mentioned and let you know my findings

Many thanks

Marcus
0
Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

Hanno P.S.IT Consultant and Infrastructure ArchitectCommented:
Starting a script from cron does NOT source the standard login files like /etc/profile ...
Therefore, make sure you use absolut path names for anything else but /usr/bin
0
Brian UtterbackPrinciple Software EngineerCommented:
There are only two command in his example, rm and flar. If flar wasn't found, then there would not be any
extra processes hanging around.  It would be a bad thing if the flar script itself needed to have the path
set a certina way though.
0
Hanno P.S.IT Consultant and Infrastructure ArchitectCommented:
blu,
I think the more importatn part is capturing the command's output for further analysis
0
omarfaridCommented:
If you check your mail on the system, do you see error messages from the cron job?

if not redirected, all commands output will be send by mail to user.

you may set env variables needed by the commands in the script itself.
0
marcusjonesAuthor Commented:
Actually, I've looked and looked at this script and I can't see anything wrong. I now think it's actually the sftp script that is causing the problem

The cron environment is as follows:

HOME=/
LOGNAME=root
PATH=/usr/sbin:/usr/bin
SHELL=/usr/bin/sh
TZ=GMT0

I can't see that there would be any problem with paths to the required binaries...

I reviewed the flar script yesterday and ran it under cron every how and it seem that all processes terminated properly

However, I set the sftp script to copy a flar image of another server at midnight and came in today to find about 60 SSH processes and 260 or so SFTP processes!

I'm wondering now whether I need to update that script so that the FTP commands are in a separate file - I saw this on the following web page

http://www.snailbook.com/faq/scripting-sftp.auto.html

But I still don't understand why that would be required!

I will updload the sftpCopy script and the log file

Any guidance welcome and if this needs to be opened as a new question, please let me know

Many thanks

Marcus
#!/bin/sh
 
# Set environment variables
 
remoteHost='server2'
timestamp=`date +'%Y%m%d%H%M'`
 
 
# Pull file from Remote System
 
cd /export/home/backups/remote
sftp $remoteHost <<EOF
cd /export/home/backups/local
get backup.flar
quit
EOF
 
 
# Rename archive
 
mv backup.flar $remoteHost-backup-$timestamp.flar
 
 
# Remove the old archive
 
lsNumber=`ls | wc -l`
 
if [ $lsNumber -gt 1 ]; then
       rm `ls -1t | tail -1`
   exit 1
fi

Open in new window

sftpCopy.log
0
Hanno P.S.IT Consultant and Infrastructure ArchitectCommented:
why don't you use scp instead of sftp?
This will not require tweaking the interactive part


#!/bin/sh
 
# Set environment variables
remoteHost='server2'
 
# Put file from onto "remoteHost" System (one line!):
scp /export/home/backups/remote/backup.flar myuser@$remoteHost:/export/home/backups/local/$remoteHost-backup-`date +'%Y%m%d%H%M'`.flar
 
# Remove the oldest archive
# NOTE: ls -"one" | wc -"ell" -gt "one" AND ls -"one"t, tail -"ell"
[ `ls -1 | wc -l` -gt 1 ] && rm `ls -1t | tail -1`

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
marcusjonesAuthor Commented:
I tried this with your SCP script and it worked fine. It's much easier than SFTP too, so that's great! Thank you very much for your help - It's greatly appreciated, particularly as I'm so bad at writing code! I'll get there in the end perhaps...
0
Hanno P.S.IT Consultant and Infrastructure ArchitectCommented:
no problem -- just keep practicing ;-)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
System Programming

From novice to tech pro — start learning today.