AnandSahoo
asked on
Perl/Shell Script to check the disk size of destination from source and alert
Hi Team,
Have one issue and looking for a workaround.
I have following Unix Servers with OS - SunOS 5.9 Generic_122300-42 sun4u sparc SUNW,Sun-Fire-V490
Source - UXGIPR01 or UXGIPR01
There are five destination servers. as below.
UXINPR01
AAPGICP001
GCPDBS001
SOADBS001
CNASIAAPPUX001
One Java file runs on source server and creates five different log files in
uxgipr01:/u04/output> directory.
After that using FTS (File Transfer Service) each files are sent to respective servers.
Now problem is if the disk space is less in the destination servers that file is failing to copy.
I need to have one script to run prior to FTP which can log in to the destination with credentials (If required) and alert/send email if the destination servers doesnt have enough space.
You can suggest other ideas also to achieve this. Early response wil be appreciated.
Have one issue and looking for a workaround.
I have following Unix Servers with OS - SunOS 5.9 Generic_122300-42 sun4u sparc SUNW,Sun-Fire-V490
Source - UXGIPR01 or UXGIPR01
There are five destination servers. as below.
UXINPR01
AAPGICP001
GCPDBS001
SOADBS001
CNASIAAPPUX001
One Java file runs on source server and creates five different log files in
uxgipr01:/u04/output> directory.
After that using FTS (File Transfer Service) each files are sent to respective servers.
Now problem is if the disk space is less in the destination servers that file is failing to copy.
I need to have one script to run prior to FTP which can log in to the destination with credentials (If required) and alert/send email if the destination servers doesnt have enough space.
You can suggest other ideas also to achieve this. Early response wil be appreciated.
ASKER
I want to compare with the source file. Little detailed explanation will be helpful
1) Install the public key of the source server to all the 5 servers.
http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html
2) so that you can able to login automatically (without providing password) to all these server using ssh command
3) using the below command, you can get the disk usage of the mount point, when you gonna put the file.
ssh user@server "du -h /mount | grep "mount-nam" # take the free space ( by cutting or using awk )
4) once you are happy with the space of that server, do scp of the log file
http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html
2) so that you can able to login automatically (without providing password) to all these server using ssh command
3) using the below command, you can get the disk usage of the mount point, when you gonna put the file.
ssh user@server "du -h /mount | grep "mount-nam" # take the free space ( by cutting or using awk )
4) once you are happy with the space of that server, do scp of the log file
ASKER
Yes. We can do this in this way.
I was looking to automate this with a script. Which can
compare the files with disk space and SENDEMAIL.
Schedule it to run before ftp job.
I was looking to automate this with a script. Which can
compare the files with disk space and SENDEMAIL.
Schedule it to run before ftp job.
Here you go, this will do what you need. Cron it before your FTP script runs, and make sure that you have the passwordless (key-based) ssh set up first.
#!/bin/sh
SERVERS="UXINPR01 AAPGICP001 GCPDBS001 SOADBS001 CNASIAAPPUX001"
EMAILRCPT="you@youraddress.com"
FILE_TO_COPY="/path/to/myfile/toftp"
MYFILESIZE=`du -k $FILE_TO_COPY | awk '{print $1}'`
for SERVER in $SERVERS;
do
KFREE=`ssh $SERVER df -k | grep /u04 | awk '{print $4}'`
if [ $MYFILESIZE -gt $KFREE ]; then
mail -s "Server $SERVER has $KFREE available, needs $MYFILESIZE" $EMAILRCPT < /dev/null
fi
done
ASKER
Few questions.
1.i have 5 different servers .will it show the diskpace output separately for 5 servers ?
2. In the source directory we have 5 different files. Each meant for one server. Will it check the each file size or entire directory size ?
3. Is there any setup required for email?
Little more details about the functionality will be helpful.
1.i have 5 different servers .will it show the diskpace output separately for 5 servers ?
2. In the source directory we have 5 different files. Each meant for one server. Will it check the each file size or entire directory size ?
3. Is there any setup required for email?
Little more details about the functionality will be helpful.
It would probably be easier to just repost with comments on how it works, but briefly:
1. Yes, it goes to each (see $SERVERS)
2. It checks the whole directory, but we can modify that if needed
3. Generally, no, most any Unix server can mail without any setup - if your source server is also Solaris 9, it shouldn't be a problem.
1. Yes, it goes to each (see $SERVERS)
2. It checks the whole directory, but we can modify that if needed
3. Generally, no, most any Unix server can mail without any setup - if your source server is also Solaris 9, it shouldn't be a problem.
#!/bin/sh
# Each of your server names
SERVERS="UXINPR01 AAPGICP001 GCPDBS001 SOADBS001 CNASIAAPPUX001"
# Email address to mail the results to if a system has too little space
EMAILRCPT="you@youraddress.com"
# This is the file or folder that you want to copy to each remote server
FILE_TO_COPY="/path/to/myfile/toftp"
# This puts the size in KB of the file/folder you want to copy into a variable
MYFILESIZE=`du -k $FILE_TO_COPY | awk '{print $1}'`
# Now we cycle through the list of server names, one by one
for SERVER in $SERVERS;
do
# ssh to the server, get the disk space in KB available on partition /u04
KFREE=`ssh $SERVER df -k | grep /u04 | awk '{print $4}'`
# See if the file we're copying is greater than the free space available
if [ $MYFILESIZE -gt $KFREE ]; then
# Mail you to let you know which server doesn't have sufficient space
mail -s "Server $SERVER has $KFREE available, needs $MYFILESIZE" $EMAILRCPT < /dev/null
fi
done
ASKER
Looks perfect .testing it now.
If i want to check individual file and respective server separately
i can declare $server1,$server2.......
$FILE_TOCOPY can the exact file name after \path
and compare separately usinf if ...DO is not required
corect me if im wrong plz
If i want to check individual file and respective server separately
i can declare $server1,$server2.......
$FILE_TOCOPY can the exact file name after \path
and compare separately usinf if ...DO is not required
corect me if im wrong plz
ASKER
Just completed testing.
Looks like the mail command is not working and I am not getting any test email.
I have also tried like this
mail -s "Test Email" Name@email.com
But did not receive any email.
Looks like the mail command is not working and I am not getting any test email.
I have also tried like this
mail -s "Test Email" Name@email.com
But did not receive any email.
ASKER
Have resolved the email and separate file issue. So please ignore the previous 2 posts.
Just one last doubt. If password less ssh can not be done (installing public key of source server), Can we embed the user name and password of each server in the script ?
Just one last doubt. If password less ssh can not be done (installing public key of source server), Can we embed the user name and password of each server in the script ?
Glad you got the last two issues resolved.
The ssh command does not take a username/password option on the command line unfortunately - it is designed that way for security reasons. There are ways around this, but I'm afraid none of them are good.
Are you having trouble putting the key on the server? Obviously you have access or you wouldn't be able to run the script, so if you're stuck, I can help you close out that last simple part.
The ssh command does not take a username/password option on the command line unfortunately - it is designed that way for security reasons. There are ways around this, but I'm afraid none of them are good.
Are you having trouble putting the key on the server? Obviously you have access or you wouldn't be able to run the script, so if you're stuck, I can help you close out that last simple part.
ASKER
Yes, If I Want to install the public key of the source server in destnations, its done by some one else and require some approval. Hence will take time for me to test in live.
As I have mentioned above I want to compare each files not the folders. File naming convention is always the same except ends with datetime.
What if the file itself is not available ?Can we tweek it to send that alert too ?
As I have mentioned above I want to compare each files not the folders. File naming convention is always the same except ends with datetime.
What if the file itself is not available ?Can we tweek it to send that alert too ?
Please give me an example of what the 5 source files look like for a given day, and I can modify the script appropriately.
And yes, if the file itself is not present, it can alert on that too, although that is not technically part of your question, so please try to keep in mind that this is not really a project-based exchange, more of a way to get you going in the right direction.
And yes, if the file itself is not present, it can alert on that too, although that is not technically part of your question, so please try to keep in mind that this is not really a project-based exchange, more of a way to get you going in the right direction.
ASKER
The file name changes except the following
for UXINPR01 startswith 01
for AAPGICP001 startswith 02
for GCPDBS001 startswith 03
for SOADBS001 starswith 04
for CNASIAAPPUX001 startswith 05
Sorry I really understand this is just to get the idea not for the complete project.
Actually I have tried this in perl script and facing similar challenge to login to the server and comparing one by one.
But your shell scripting looks portable and works well in Unix, hence decided to adopt this.
While Testing I came across this problem. What If the file itself is not available.
for UXINPR01 startswith 01
for AAPGICP001 startswith 02
for GCPDBS001 startswith 03
for SOADBS001 starswith 04
for CNASIAAPPUX001 startswith 05
Sorry I really understand this is just to get the idea not for the complete project.
Actually I have tried this in perl script and facing similar challenge to login to the server and comparing one by one.
But your shell scripting looks portable and works well in Unix, hence decided to adopt this.
While Testing I came across this problem. What If the file itself is not available.
Here is the rewrite that has newly added:
- Checks the source file individually now for size
- Alerts if the source file is not found (or if there are more than 1 file found starting with 01, 02, etc.)
- Checks the source file individually now for size
- Alerts if the source file is not found (or if there are more than 1 file found starting with 01, 02, etc.)
#!/bin/sh
# Create a counter for the source files
COUNT=1
# Each of your server names
SERVERS="UXINPR01 AAPGICP001 GCPDBS001 SOADBS001 CNASIAAPPUX001"
# Email address to mail the results to if a system has too little space
EMAILRCPT="you@youraddress.com"
# This is the folder that holds the files to be copied
SOURCE_COPY_DIR="/path/to/myfiles"
# Now we cycle through the list of server names, one by one
for SERVER in $SERVERS;
do
# This looks for a file in your source directory that starts with 0<counter value>
EXIST=`ls $SOURCE_COPY_DIR/0$COUNT* | wc -l`
if [ $EXIST -ne 1 ]; then
# The file isn't present (or there is more than 1 match)
mail -s "Error locating source file $SOURCE_COPY_DIR/0$COUNT*" $EMAILRCPT < /dev/null
else
# This puts the size in KB of the file/folder you want to copy into a variable
MYFILESIZE=`du -k $SOURCE_COPY_DIR/0$COUNT* | awk '{print $1}'`
# ssh to the server, get the disk space in KB available on partition /u04
KFREE=`ssh $SERVER df -k | grep /u04 | awk '{print $4}'`
# See if the file we're copying is greater than the free space available
if [ $MYFILESIZE -gt $KFREE ]; then
# Mail you to let you know which server doesn't have sufficient space
mail -s "Server $SERVER has $KFREE available, needs $MYFILESIZE to copy file $SOURCE_COPY_DIR/0$COUNT*" $EMAILRCPT < /dev/null
fi
fi
# Increment the counter by 1
COUNT=`expr $COUNT + 1`
done
ASKER
Im sure this will give me idea. Interms of syntax also i can now change it as per my requirement.
One last thing. Can we do anything about ssh passwordless login to avoid installing key in other servers ?
One last thing. Can we do anything about ssh passwordless login to avoid installing key in other servers ?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks a ton for your time and effort. Really appreciate.
and you can easily check the disk usage of the destination servers by using the below command
Open in new window
once you installed the source server public key into the destination servers, you can easily transfer the files using sftp or scp