Perl/Shell Script to check the disk size of destination from source and alert

Hi Team,
Have one issue and looking for a workaround.
I have following Unix Servers with OS - SunOS 5.9 Generic_122300-42 sun4u sparc SUNW,Sun-Fire-V490
Source - UXGIPR01 or UXGIPR01
There are five destination servers. as below.
UXINPR01
AAPGICP001
GCPDBS001
SOADBS001
CNASIAAPPUX001
One Java file runs on source server and creates five different log files in
uxgipr01:/u04/output> directory.
After that using FTS (File Transfer Service) each files are sent to respective servers.
Now problem is if the disk space is less in the destination servers that file is failing to copy.
I need to have one script to run prior to FTP which can log in to the destination with credentials (If required) and alert/send email if the destination servers doesnt have enough space.

You can suggest other ideas also to achieve this. Early response wil be appreciated.

AnandSahooAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Kamaraj SubramanianApplication Support AnalystCommented:
set the passwordless ssh setup ( just google it )

and you can easily check the disk usage of the destination servers by using the below command

ssh username@server "du -h /mount/name" | grep "whatever" # to find out the disk usage and free space

Open in new window


once you installed the source server public key into the destination servers, you can easily transfer the files using sftp or scp
AnandSahooAuthor Commented:
I want to compare with the source file. Little detailed explanation will be helpful
Kamaraj SubramanianApplication Support AnalystCommented:
1) Install the public key of the source server to all the 5 servers.

http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html

2) so that you can able to login automatically (without providing password) to all these server using ssh command

3) using the below command, you can get the disk usage of the mount point, when you gonna put the file.

ssh user@server "du -h /mount | grep "mount-nam"  # take the free space ( by cutting or using awk )

4) once you are happy with the space of that server, do scp of the log file

Amazon Web Services

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

AnandSahooAuthor Commented:
Yes. We can do this in this way.
 I was looking to automate this with a script. Which can
compare the files with disk space and SENDEMAIL.
Schedule it to run before ftp job.
xtermCommented:
Here you go, this will do what you need.  Cron it before your FTP script runs, and make sure that you have the passwordless (key-based) ssh set up first.
#!/bin/sh

SERVERS="UXINPR01 AAPGICP001 GCPDBS001 SOADBS001 CNASIAAPPUX001"
EMAILRCPT="you@youraddress.com"
FILE_TO_COPY="/path/to/myfile/toftp"
MYFILESIZE=`du -k $FILE_TO_COPY | awk '{print $1}'`

for SERVER in $SERVERS;
do
        KFREE=`ssh $SERVER df -k | grep /u04 | awk '{print $4}'`
        if [ $MYFILESIZE -gt $KFREE ]; then
                mail -s "Server $SERVER has $KFREE available, needs $MYFILESIZE" $EMAILRCPT < /dev/null
        fi
done

Open in new window

AnandSahooAuthor Commented:
Few questions.
1.i have 5 different servers .will it show the diskpace output separately for 5 servers ?
2. In the source directory we have 5 different files. Each meant for one server. Will it check the each file size or entire directory size ?
3. Is there any setup required for email?

Little more details about the functionality will be helpful.
xtermCommented:
It would probably be easier to just repost with comments on how it works, but briefly:

1.  Yes, it goes to each (see $SERVERS)
2.  It checks the whole directory, but we can modify that if needed
3.  Generally, no, most any Unix server can mail without any setup - if your source server is also Solaris 9, it shouldn't be a problem.
#!/bin/sh

# Each of your server names
SERVERS="UXINPR01 AAPGICP001 GCPDBS001 SOADBS001 CNASIAAPPUX001"
# Email address to mail the results to if a system has too little space
EMAILRCPT="you@youraddress.com"
# This is the file or folder that you want to copy to each remote server
FILE_TO_COPY="/path/to/myfile/toftp"
# This puts the size in KB of the file/folder you want to copy into a variable
MYFILESIZE=`du -k $FILE_TO_COPY | awk '{print $1}'`

# Now we cycle through the list of server names, one by one
for SERVER in $SERVERS;
do
        # ssh to the server, get the disk space in KB available on partition /u04
        KFREE=`ssh $SERVER df -k | grep /u04 | awk '{print $4}'`
        # See if the file we're copying is greater than the free space available
        if [ $MYFILESIZE -gt $KFREE ]; then
                # Mail you to let you know which server doesn't have sufficient space
                mail -s "Server $SERVER has $KFREE available, needs $MYFILESIZE" $EMAILRCPT < /dev/null
        fi
done

Open in new window

AnandSahooAuthor Commented:
Looks perfect .testing it now.

If i want to check individual file and respective server separately
i can declare $server1,$server2.......
$FILE_TOCOPY can the exact file name after \path
and compare separately usinf if ...DO is not required
corect me if im wrong plz
AnandSahooAuthor Commented:
Just completed testing.
Looks like the mail command is not working and I am not getting any test email.

I have also tried like this
mail -s "Test Email" Name@email.com

But did not receive any email.
AnandSahooAuthor Commented:
Have resolved the email and separate file issue. So please ignore the previous 2 posts.

Just one last doubt. If password less ssh can not be done (installing public key of source server), Can we embed the user name and password of each server in the script ?
xtermCommented:
Glad you got the last two issues resolved.

The ssh command does not take a username/password option on the command line unfortunately - it is designed that way for security reasons.  There are ways around this, but I'm afraid none of them are good.

Are you having trouble putting the key on the server?  Obviously you have access or you wouldn't be able to run the script, so if you're stuck, I can help you close out that last simple part.
AnandSahooAuthor Commented:
Yes, If I Want to install the public key of the source server in destnations, its done by some one else and require some approval. Hence will take time for me to test in live.

As I have mentioned above I want to compare each files not the folders. File naming convention is always the same except ends with datetime.
What if the file itself is not available ?Can we tweek it to send that alert too ?
xtermCommented:
Please give me an example of what the 5 source files look like for a given day, and I can modify the script appropriately.

And yes, if the file itself is not present, it can alert on that too, although that is not technically part of your question, so please try to keep in mind that this is not really a project-based exchange, more of a way to get you going in the right direction.
AnandSahooAuthor Commented:
The file name changes except the following
for UXINPR01 startswith 01
for AAPGICP001 startswith 02
for GCPDBS001 startswith 03
for SOADBS001 starswith 04
for CNASIAAPPUX001 startswith 05

Sorry I really understand this is just to get the idea not for the complete project.
Actually I have tried this in perl script and facing similar challenge to login to the server and comparing one by one.
But your shell scripting looks portable and works well in Unix, hence decided to adopt this.

While Testing I came across this problem. What If the file itself is not available.

xtermCommented:
Here is the rewrite that has newly added:
- Checks the source file individually now for size
- Alerts if the source file is not found (or if there are more than 1 file found starting with 01, 02, etc.)


#!/bin/sh

# Create a counter for the source files
COUNT=1
# Each of your server names
SERVERS="UXINPR01 AAPGICP001 GCPDBS001 SOADBS001 CNASIAAPPUX001"
# Email address to mail the results to if a system has too little space
EMAILRCPT="you@youraddress.com"
# This is the folder that holds the files to be copied
SOURCE_COPY_DIR="/path/to/myfiles"

# Now we cycle through the list of server names, one by one
for SERVER in $SERVERS;
do
        # This looks for a file in your source directory that starts with 0<counter value>
        EXIST=`ls $SOURCE_COPY_DIR/0$COUNT* | wc -l`
        if [ $EXIST -ne 1 ]; then
                # The file isn't present (or there is more than 1 match)
                mail -s "Error locating source file $SOURCE_COPY_DIR/0$COUNT*" $EMAILRCPT < /dev/null
        else
                # This puts the size in KB of the file/folder you want to copy into a variable
                MYFILESIZE=`du -k $SOURCE_COPY_DIR/0$COUNT* | awk '{print $1}'`
                # ssh to the server, get the disk space in KB available on partition /u04
                KFREE=`ssh $SERVER df -k | grep /u04 | awk '{print $4}'`
                # See if the file we're copying is greater than the free space available
                if [ $MYFILESIZE -gt $KFREE ]; then
                        # Mail you to let you know which server doesn't have sufficient space
                        mail -s "Server $SERVER has $KFREE available, needs $MYFILESIZE to copy file $SOURCE_COPY_DIR/0$COUNT*" $EMAILRCPT < /dev/null
                fi
        fi
        # Increment the counter by 1
        COUNT=`expr $COUNT + 1`
done

Open in new window

AnandSahooAuthor Commented:
Im sure this will give me idea. Interms of syntax also i can now change it as per my requirement.

One last thing. Can we do anything about ssh passwordless login to avoid installing key  in other servers ?
xtermCommented:
I'm not sure exactly why this is such a big deal to enable - all you have to do is copy your ~/.ssh/id_rsa.pub file and paste it into ~/.ssh/authorized_keys on the 5 servers.  Should take you 5 minutes.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
AnandSahooAuthor Commented:
Thanks a ton for your time and effort. Really appreciate.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Perl

From novice to tech pro — start learning today.