shweta agarwal
asked on
Fetching data from Multiple Linux Server using Shell Scripting
I am working Linux server using Shell Scripting and executing the commands on multiple Linux servers using SSH but I am not able to fetch the output in the text file. Please, help me out.
Your ssh command is writing all of the output to abc.txt on the remote machine. If you want the data returned to the local machine, just send it to stdout and display that locally:
printf "%s\\n" "$(uname)"
ssh -T $remote_srv <<\EOF
var1=$(df as before)
var2=$(df as before)
printf "$(uname)\\n"
echo $var1
echo $var2
EOF
By the way, you shouldn't include variable data in the format part of a printf - that may allow code insertion. Instead specify the format explicitly, and insert data into that format, as in:printf "%s\\n" "$(uname)"
ASKER
So, please tell me how to fetch the disk free usage from the multiple Linux servers using shell scripting. The servers name should be in the text file and the output should ne store in the text file and the format of the output like this:
Server Name(Linux)
/ /etc /tmp
20% 4.5% 10%
the SSH should be used for the password less logins and the output file stores in the server from where the scripting is executing.
Please, help me out as soon as.
Thanks in advance :)
Server Name(Linux)
/ /etc /tmp
20% 4.5% 10%
the SSH should be used for the password less logins and the output file stores in the server from where the scripting is executing.
Please, help me out as soon as.
Thanks in advance :)
Add echo $(hostname) to the commands executed on the remote site just in case the server name used in the list is an alias. To maintain an old reference, a CNAME record was created in DNS.
Though before issuing the ssh you can add echo "running commands on server: $s
I find peculiar that you are running multiple loops:
One to build the list of servers, and the inner when you actually connect and execute commands on the remote_srv..seemingly as many times as the length of your server name.
You execute ssh the sum of number of servers in servers.txt times the length of servername.
The other part, you define a function header but you pass it a generic, "Disk Usage" instead of "Disk a Usage On $remote_srv"
Though before issuing the ssh you can add echo "running commands on server: $s
I find peculiar that you are running multiple loops:
One to build the list of servers, and the inner when you actually connect and execute commands on the remote_srv..seemingly as many times as the length of your server name.
You execute ssh the sum of number of servers in servers.txt times the length of servername.
The other part, you define a function header but you pass it a generic, "Disk Usage" instead of "Disk a Usage On $remote_srv"
ASKER
can you show the proper code to execute?
Can you post your code in plain text, versus as an image?
Have you incorporated Simon's suggestion?
eplace header "Disk Usage" with
Header "Disk Usage on $(remote_srv)"
First thing is to get the output back.
To reformat the data from
Mountpoint usage...
To the other .........
Involves using the mount point as the header and the available space as the next row ........
Have you incorporated Simon's suggestion?
eplace header "Disk Usage" with
Header "Disk Usage on $(remote_srv)"
First thing is to get the output back.
To reformat the data from
Mountpoint usage...
To the other .........
Involves using the mount point as the header and the available space as the next row ........
ASKER
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}= ========== ========== =====#"
}
n=${#remote_srv[@]}
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
var1=$(df -h|awk '{print $6 "\t"}')
var2=$(df -h|awk '{print $5 "\t"}')
printf "$(uname)\\n" >> abc.txt
echo $var1 >> abc.txt
echo $var2 >> abc.txt
echo $'\n' >> abc.txt
EOF
done
done
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}=
}
n=${#remote_srv[@]}
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
var1=$(df -h|awk '{print $6 "\t"}')
var2=$(df -h|awk '{print $5 "\t"}')
printf "$(uname)\\n" >> abc.txt
echo $var1 >> abc.txt
echo $var2 >> abc.txt
echo $'\n' >> abc.txt
EOF
done
done
Try the following,
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}= ========== ========== =====#"
}
n=${#remote_srv[@]}
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
var1=$(df -h|awk '{print $6 "\t"}').
var2=$(df -h|awk '{print $5 "\t"}')
printf "%s\t%s\n",$(hostname),$(u name)
echo $var1
echo $var2
echo $'\n'
EOF
done #end server list loop.
When calling the script redirect all output to file
./script >abc.txt
Or
./script >>abc.txt
If you want to continually collect data?
Run the script with fewer entries in servers.txt...
Depending on what you want the information for, using SNMP polling might be an option to consider.
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}=
}
n=${#remote_srv[@]}
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
var1=$(df -h|awk '{print $6 "\t"}').
var2=$(df -h|awk '{print $5 "\t"}')
printf "%s\t%s\n",$(hostname),$(u
echo $var1
echo $var2
echo $'\n'
EOF
done #end server list loop.
When calling the script redirect all output to file
./script >abc.txt
Or
./script >>abc.txt
If you want to continually collect data?
Run the script with fewer entries in servers.txt...
Depending on what you want the information for, using SNMP polling might be an option to consider.
ASKER
It is not working as I want the output in the file. I need only the output with which server it is connected and for the fisrt server it is not giving me the server name and hostname. Please, recodify it or tell me the other solution.
Thank you :)
Thank you :)
What do you get as a response from that server?
Please post the samp,e output your script generates.
Please post the samp,e output your script generates.
To maintain formating
try this
The variable assignment as you were doing effectively was responsible for trimming the space ......
try this
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}==========================#"
}
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date_info) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
printf "%s\t%s\n",$(/bin/hostname),$(uname)
$(df -h|awk '{printf "%10s\t",$6 }').
$(df -h|awk '{print "%10f\t",$5 }')
echo $'\n'
EOF
done #end server list loop.
The variable assignment as you were doing effectively was responsible for trimming the space ......
ASKER
It is not working fine and even not storing the output on the file and at least the previous code was storing the output in the file.
If do you have the solution regarding my query then please send me that piece of code according to you and again I am telling my requirements what I need to do as below:-
1).I have list of server names in a text file.
2).I need to fetch the disk free usage from each of the server using SSH in the given format.
Server name (on which it is executed)
/ /etc /tmp
10% 13% 17%
3).I need to store the data in the file of each server on which it is executed only.
Please help me out soon bcz I need it urgently.
If do you have the solution regarding my query then please send me that piece of code according to you and again I am telling my requirements what I need to do as below:-
1).I have list of server names in a text file.
2).I need to fetch the disk free usage from each of the server using SSH in the given format.
Server name (on which it is executed)
/ /etc /tmp
10% 13% 17%
3).I need to store the data in the file of each server on which it is executed only.
Please help me out soon bcz I need it urgently.
The script provides the output on screen, add the redirect (>> somelocalfile) to file when the script is executed.
Add back the >> abc.txt you had before.
I am confused, you wanted the data on the server where you run your aggregating script.
If you need a copy on the remote and data sent back, double up on the commands.
Add back the >> abc.txt you had before.
I am confused, you wanted the data on the server where you run your aggregating script.
If you need a copy on the remote and data sent back, double up on the commands.
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}==========================#"
}
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date_info) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
printf "%s\t%s\n",$(/bin/hostname),$(uname)
$(df -h|awk '{printf "%10s\t",$6 }').
$(df -h|awk '{print "%10f\t",$5 }')
echo $'\n'
#the below will store the data in a local file on each system
printf "%s\t%s\n",$(/bin/hostname),$(uname) >> abc.txt
$(df -h|awk '{printf "%10s\t",$6 }') >> abc.txt
$(df -h|awk '{print "%10f\t",$5 }') >> abc.txt
echo $'\n' >> abc.txt
EOF
done #end server list loop.
ASKER
I think you are not getting what I actually want to do in the script. By the way Thank you so much for your help.
You are running a script on System A that connects to multiple other systems. You want to see the responses from all the other systems on System A.
You could using your old script cat the abc.txt file.
Note that since you are using an append (>>) the local abc.txt will contain all prior data entries.
If the above is not what you want, provide an example of what you want. use an example as what ....... would be on one or two sample systems locally and what you want to get back to the server where you script runs. (uname identifies the type of system not its name, hostname returns the name of the system on which the command was run)
The earlier example returns all data back to the system A where the shell script is executed. The last one, does the same, but includes the local copy of the disk usage appended to the abc.txt as you had before.
Are you running this data collection script from cron, add
* x y * * /run/some/script/disk_usag e.sh >> /some/path/responses_from_ script_and _all_syste ms.txt 2>> /some/path/errorresponses
Or you can leave the error redirect and receive the errors in the email on the day they occurred......
You could using your old script cat the abc.txt file.
Note that since you are using an append (>>) the local abc.txt will contain all prior data entries.
If the above is not what you want, provide an example of what you want. use an example as what ....... would be on one or two sample systems locally and what you want to get back to the server where you script runs. (uname identifies the type of system not its name, hostname returns the name of the system on which the command was run)
The earlier example returns all data back to the system A where the shell script is executed. The last one, does the same, but includes the local copy of the disk usage appended to the abc.txt as you had before.
Are you running this data collection script from cron, add
* x y * * /run/some/script/disk_usag
Or you can leave the error redirect and receive the errors in the email on the day they occurred......
ASKER
I think you are not getting me and let me try once to explain you in the other format.
Suppose I have a Linux machine A and multiple B-F Linux machines and I want to execute the script from the machine A and it will gather the Disk Usage from the B-F machine and saves the output in a text file on the machine A. the machine A only saves the output of those machine on which the script is executed successfully in only that format I have shown you in the above comments and I don't need to provide the password for every machine B-F.
Suppose I have a Linux machine A and multiple B-F Linux machines and I want to execute the script from the machine A and it will gather the Disk Usage from the B-F machine and saves the output in a text file on the machine A. the machine A only saves the output of those machine on which the script is executed successfully in only that format I have shown you in the above comments and I don't need to provide the password for every machine B-F.
ASKER
This script is giving me the output from different servers of Linux in the text file on the local Linux server but now the problem is that it is not taking the whole data from the each server like:-
Giving the output of \ \etc \tmp
10% 13%
not giving for the "\" and from the other server it giving like :
\ \etc \tmp
10% 17% 11% /mount /bin 78% /usr
So, Please help me out to bring the whole output.
Waiting for your positive response.
Giving the output of \ \etc \tmp
10% 13%
not giving for the "\" and from the other server it giving like :
\ \etc \tmp
10% 17% 11% /mount /bin 78% /usr
So, Please help me out to bring the whole output.
Waiting for your positive response.
You are not providing me with a complete picture on whether the issue is with the execution of the commands on the remote side or whether it is the data received on the Server A.
Your issue is that your "disk" item number 1 in the list
/path/to/where/thedisk/is
causes the line to shift
/dev/sda1
or do you have mdadm, lvm overlays?
the above will output the old stuff on each server
But the information you will get on Server A is the entire df -h output as a reference so you can see whether some servers have no issues since their output has everything on one line
1 2 3 4 5 6
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 500 125 375 25% /
/dev/mapper/vg-somegroup-l ogical
25000 12500 12500 50% /home
note the LVM position 1 is on a line by itself while the others are on a line below shifted meaning instead of 1-6, since the default awk is to split on white space, there are only 6 elements, 1-5 instead of the 7 1-7 in the expected ..
made adjustments to only report the last item in the line and the one before it on the second line while excluding the header line....
do you really want the Use% to be in floating point display, ?
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}==========================#"
}
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date_info) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
printf "%s\t%s\n",$(/bin/hostname),$(uname)
$(df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }')
echo
$(df -h| grep -v 'Mounted' | awk ' (NF>1) {printf "%10s\t",$(NF-1)}')
echo $'\n'
#the below will store the data in a local file on each system
printf "%s\t%s\n",$(/bin/hostname),$(uname) >> abc.txt
$(df -h| grep -v 'Mounted'|awk ' (NF >1 ) {printf "%10s\t",$(NF) }') >> abc.txt
echo >> abc.txt
$(df -h| grep -v 'Mounted'|awk ' (NF>1) { printf "%10s\t",$(NF-1)}') >> abc.txt
echo $'\n' >> abc.txt
EOF
done #end server list loop.
Your issue is that your "disk" item number 1 in the list
/path/to/where/thedisk/is
causes the line to shift
/dev/sda1
or do you have mdadm, lvm overlays?
the above will output the old stuff on each server
But the information you will get on Server A is the entire df -h output as a reference so you can see whether some servers have no issues since their output has everything on one line
1 2 3 4 5 6
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 500 125 375 25% /
/dev/mapper/vg-somegroup-l
25000 12500 12500 50% /home
note the LVM position 1 is on a line by itself while the others are on a line below shifted meaning instead of 1-6, since the default awk is to split on white space, there are only 6 elements, 1-5 instead of the 7 1-7 in the expected ..
made adjustments to only report the last item in the line and the one before it on the second line while excluding the header line....
do you really want the Use% to be in floating point display, ?
ASKER
I am attaching the output with this. So, please have a look on it and suggest me the alternative way to do that.
output.zip
output.zip
I made an adjustment in the updated script included in the last comment.
The hostname, system type using printf ..... not working as expected
The hostname, system type using printf ..... not working as expected
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}==========================#"
}
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date_info) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
echo -e "$(/bin/hostname)\n$(uname)\n"
$(df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }')
echo
$(df -h| grep -v 'Mounted' | awk ' (NF>1) {printf "%10s\t",$(NF-1)}')
echo $'\n'
#the below will store the data in a local file on each system
echo -e "$(/bin/hostname)\n$(uname)\n" >> abc.txt
$(df -h| grep -v 'Mounted'|awk ' (NF >1 ) {printf "%10s\t",$(NF) }') >> abc.txt
echo >> abc.txt
$(df -h| grep -v 'Mounted'|awk ' (NF>1) { printf "%10s\t",$(NF-1)}') >> abc.txt
echo $'\n' >> abc.txt
EOF
done #end server list loop.
I think the script is getting a bit more complicated than it needs to be.
Also commands like:
$(df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }')
will run the command within the $(), return the output as a single line of text, then try to execute the resulting line. I think you need an "echo " in front of it just to get the data, so
echo $(df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }')
If the intention is to retrieve data from all of the remote servers so that it can be analysed or stored, then adding formatting strings (to make some of the text green, for example) will be a distraction. The inner "for" loop (the bit which outputs "Fetching info" etc) is not needed - you only have one remote server name in $remote_srv at that point.
We also haven't talked about connectivity. If they aren't already done, you need ssh keys to be set up so that passwordless login is possible.
- First check in ~/.ssh on server A (in the home directory of the user running the script) to see whether there are id_rsa or id_dsa private key files (and their corresponding id_rsa.pub or id_dsa.pub public keys). If not, create them by running "ssh-keygen", and just pressing enter for all of the questions. This will create a private/public keypair with no passphrase.
- Then for each of server B to F, try to ssh in. If it lets you in without a password being required, you are ready to go. If it asks for a password, log out of that server (so back to server A) and run "ssh-copy-id remote.host.name" and enter the password when prompted. If you then try "ssh remote.host.name", it should log you on without a password.
Now for the script. Do you really need the mountpoints listed on one line, and the percentage usage on the next? If so, Arnold's $(df...) construction seems a good way to do it. If not, just run "df" and capture the entire output.
Data on one line:
Normal "df" output:
For each one, run the script as
script_name.sh > abc.txt
Also commands like:
$(df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }')
will run the command within the $(), return the output as a single line of text, then try to execute the resulting line. I think you need an "echo " in front of it just to get the data, so
echo $(df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }')
If the intention is to retrieve data from all of the remote servers so that it can be analysed or stored, then adding formatting strings (to make some of the text green, for example) will be a distraction. The inner "for" loop (the bit which outputs "Fetching info" etc) is not needed - you only have one remote server name in $remote_srv at that point.
We also haven't talked about connectivity. If they aren't already done, you need ssh keys to be set up so that passwordless login is possible.
- First check in ~/.ssh on server A (in the home directory of the user running the script) to see whether there are id_rsa or id_dsa private key files (and their corresponding id_rsa.pub or id_dsa.pub public keys). If not, create them by running "ssh-keygen", and just pressing enter for all of the questions. This will create a private/public keypair with no passphrase.
- Then for each of server B to F, try to ssh in. If it lets you in without a password being required, you are ready to go. If it asks for a password, log out of that server (so back to server A) and run "ssh-copy-id remote.host.name" and enter the password when prompted. If you then try "ssh remote.host.name", it should log you on without a password.
Now for the script. Do you really need the mountpoints listed on one line, and the percentage usage on the next? If so, Arnold's $(df...) construction seems a good way to do it. If not, just run "df" and capture the entire output.
Data on one line:
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
echo $remote_srv
ssh -T "$remote_srv" <<\EOF
echo $(df -h|grep -v 'Mounted' |awk '{if (NF>1 ) {printf "%10s\t",$NF }}')
echo $(df -h|grep -v 'Mounted' |awk '{if (NF>1 ) {printf "%10s\t",$(NF-1) }}')
EOF
done
Normal "df" output:
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
echo $remote_srv
ssh -T "$remote_srv" df
done
For each one, run the script as
script_name.sh > abc.txt
ASKER
Hey,
have you checked my output and I think now you get the point what I actually want to do in my script. I also want the passwordless login and data of each server should be store on the server from where the script is executed.
have you checked my output and I think now you get the point what I actually want to do in my script. I also want the passwordless login and data of each server should be store on the server from where the script is executed.
Hi Shweta, The ssh key part should give you passwordless login, and the two scripts will give you"all data on one line" and "normal df output" versions. The redirection shown at the bottom of my message will store the output on the server where you are running the script.
ASKER
The header with "Data on one line" script is tested on your environment and this will provide me the passwordless login on each server?
Not sure to whom you responded.
Remove the $() surrounding the commands to be executed on the remote servers.
You needed the $() because you were assigning the results originally to a variable.
As far as password less, on the local server run ssh-keygen -dsa|rsa to generate the local user key pair. This will create an id_dsa/Id_dsa.pub or id_rsa/id_rsa.pub in ~/.ssh depending which or both with each promoting for key length ......
make sure your remote servers /etc/ssh/sshd_config have the passwordless option activated.
Once verified, you need to transfer the .pub files to the remote systems and append them into the ~/.ssh/authorized_keys file
This is al it takes for passwordless password .....
Remove the $() surrounding the commands to be executed on the remote servers.
You needed the $() because you were assigning the results originally to a variable.
As far as password less, on the local server run ssh-keygen -dsa|rsa to generate the local user key pair. This will create an id_dsa/Id_dsa.pub or id_rsa/id_rsa.pub in ~/.ssh depending which or both with each promoting for key length ......
make sure your remote servers /etc/ssh/sshd_config have the passwordless option activated.
Once verified, you need to transfer the .pub files to the remote systems and append them into the ~/.ssh/authorized_keys file
This is al it takes for passwordless password .....
ASKER
Thank you Arnold. I asked with the Simon regarding the script, that which script I will use to get the perfect output from the multiple Linux server as I want in the desired output way.
Edit the script you already have remove the $() surrounding the df commands,
Run it against one remote server and see if the results are what you are looking for.
Post the output of it is not.
Have you considered using cacti (cacti.net) to have a graphical representation of many thinks you can monitor on a server?
Does your environment includes monitoring/alerting setup?
Run it against one remote server and see if the results are what you are looking for.
Post the output of it is not.
Have you considered using cacti (cacti.net) to have a graphical representation of many thinks you can monitor on a server?
Does your environment includes monitoring/alerting setup?
There are other options too - zabbix is a powerful monitoring tool which doesn't take a lot of setting up.
ASKER
I don't want to use any tool. How to use the cacti.net in shell scripting can you explain me that? Please, post the final script which I need to run on the Linux server.
What is the point of the shell script? What happens with the data once accessed? Cacti, zabbix, etc. could provide you with proactive notifications or graphical/visual indicators ..............
Also, enabling SNMP on the servers if not already could simplify data extraction from using ssh, to pulling the info using snmpget
Also, enabling SNMP on the servers if not already could simplify data extraction from using ssh, to pulling the info using snmpget
ASKER
I don't want to use the tool but you can tell me how to use this tool for fetching data from multiple Linux servers.
The tool, cacti, zabbix, once setup will poll the systems at a regular interval without any additional intervention from you.
What about using the posted script ?
What about using the posted script ?
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
header() {
echo "#==================={$@}==========================#"
}
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
echo "$(tput bold)$(tput setaf 2)* Connected Server: ${remote_srv[$i]} @@ $(date_info) @@"
echo "* Fetching info...$(tput sgr 0)"
s=${remote_srv[$i]}
done # end loop for length of servername
header "Disk Usage on $remote_srv "
ssh -T $remote_srv<<\EOF
echo -e "$(/bin/hostname)\n$(uname)\n"
df -h|grep -v 'Mounted' |awk ' (NF>1 ) {printf "%10s\t",$NF }'
echo
df -h| grep -v 'Mounted' | awk ' (NF>1) {printf "%10s\t",$(NF-1)}'
echo $'\n'
#the below will store the data in a local file on each system
echo -e "$(/bin/hostname)\n$(uname)\n" >> abc.txt
df -h| grep -v 'Mounted'|awk ' (NF >1 ) {printf "%10s\t",$(NF) }'>> abc.txt
echo >> abc.txt
df -h| grep -v 'Mounted'|awk ' (NF>1) { printf "%10s\t",$(NF-1)}' >> abc.txt
echo $'\n' >> abc.txt
EOF
done #end server list loop.
ASKER
Till now no body hadn't provided me the right solution to do this. So, please provide me the right and suitable script to do this.
What is the issue with the last attempt?
Hopefully you understand that it is impossible for those of us without access to understand what you are seeing that you do not like.
Hopefully you understand that it is impossible for those of us without access to understand what you are seeing that you do not like.
ASKER
The below is the output that I am getting in not the desired format. So, please suggest me the solution so, that I can get the output in the desired way as I want and also wants to add one thing that it is not collecting the whole data from the servers.
testvm.com,Linux
Mounted/dev/shm/boot
Use% / 1% 33%
vpl69.com,Linux
Mounted/dev/shm/boot/expor t/linux/Mo ndo-Images
Use% / 0% 13% /home /opt /tmp /usr /var 72% 75% /usr/oracle /opt/netvantage /Mondo-Images1 /mnt/nmon_data
testvm.com,Linux
Mounted/dev/shm/boot
Use% / 1% 33%
vpl69.com,Linux
Mounted/dev/shm/boot/expor
Use% / 0% 13% /home /opt /tmp /usr /var 72% 75% /usr/oracle /opt/netvantage /Mondo-Images1 /mnt/nmon_data
Not sure what's happening here. Please run the script without the "awk" reformatting, so that we can see what is going on with "df".
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
echo Host: $remote_srv at $(date '+%Y-%m-%e %H:%M:%S')
ssh -T $remote_srv<<\EOF
hostname
uname
df -h
EOF
done
It may be better to collect all of the information from the servers (i.e. the entire "df" output), the postprocess it on the server that the script is running on.
You seem to be focused on the formatting before answering whether the data within is what is needed/wanted.
Usually, one makes sure the data they need is present, the formatting is an esthetic and provides no useful information, it just makes it "easier" on the eyes.
Does the example obtain the required information from the remote server?
Please post the output you receive. and are you using the last script example I posted or are you using something else?
I.e. which script are you using and what output are you getting?
Usually, one makes sure the data they need is present, the formatting is an esthetic and provides no useful information, it just makes it "easier" on the eyes.
Does the example obtain the required information from the remote server?
Please post the output you receive. and are you using the last script example I posted or are you using something else?
I.e. which script are you using and what output are you getting?
ASKER
Hi, I am posting the script and the output as well as.
This is the script that I am running:-
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
s=${remote_srv[$i]}
done # end loop for length of servername
ssh -T $remote_srv<<\EOF
printf "%s\t%s\n",$(/bin/hostname ),$(uname)
echo $'\n'
var1=$(df -h|awk '{printf "%10s\t",$6 }')
var2=$(df -h|awk '{print "%10s\t",$5 }')
echo $var1
echo $var2
echo $'\n'
EOF
done
and this is the output that I am getting, which is not in desired way and incomplete information.
testvm.com,Linux
Mounted/dev/shm/boot
Use% / 1% 33%
vpl69.com,Linux
Mounted/dev/shm/boot/expor t/linux/Mo ndo-Images
Use% / 0% 13% /home /opt /tmp /usr /var 72% 75% /usr/oracle /opt/netvantage /Mondo-Images1 /mnt/nmon_data
This is the script that I am running:-
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
n=${#remote_srv[@]}
date_info=$(/bin/date +"%Y-%m-%e %H:%M:%S")
for (( i=0;i<$n;i++)); do
echo -e
s=${remote_srv[$i]}
done # end loop for length of servername
ssh -T $remote_srv<<\EOF
printf "%s\t%s\n",$(/bin/hostname
echo $'\n'
var1=$(df -h|awk '{printf "%10s\t",$6 }')
var2=$(df -h|awk '{print "%10s\t",$5 }')
echo $var1
echo $var2
echo $'\n'
EOF
done
and this is the output that I am getting, which is not in desired way and incomplete information.
testvm.com,Linux
Mounted/dev/shm/boot
Use% / 1% 33%
vpl69.com,Linux
Mounted/dev/shm/boot/expor
Use% / 0% 13% /home /opt /tmp /usr /var 72% 75% /usr/oracle /opt/netvantage /Mondo-Images1 /mnt/nmon_data
But we don't know what your "df" output looks like. Please run my simplified version to get the df output without any processing. then we will have a better chance of formatting it as you want.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I am not getting it in the right way. So, please write the whole code for my understandability.
For now, please just run df -h with no changes:
#!/bin/bash
for remote_srv in $( cat servers.txt ); do
echo Host: $remote_srv at $(date '+%Y-%m-%e %H:%M:%S')
ssh -T $remote_srv<<\EOF
hostname
uname
df -h
EOF
done
Once we've worked out what the output looks like, we can start applying formatting.
The formatting of df is altered because you are assigning the data to a variable.
Instead of
var1=$(df -h|awk '{printf "%10s\t",$6 }')
var2=$(df -h|awk '{print "%10s\t",$5 }')
echo $var1
echo $var2
echo $'\n'
Just have
df -h|awk '{printf "%10s\t",$6 }'
df -h|awk '{print "%10s\t",$5 }'
echo $'\n'
see the last comment with code that I posted.
Instead of
var1=$(df -h|awk '{printf "%10s\t",$6 }')
var2=$(df -h|awk '{print "%10s\t",$5 }')
echo $var1
echo $var2
echo $'\n'
Just have
df -h|awk '{printf "%10s\t",$6 }'
df -h|awk '{print "%10s\t",$5 }'
echo $'\n'
see the last comment with code that I posted.
ASKER
After which command i need to use which command please tell me.. i really need this so help me out.
Please look at your script find the first pattern, replace with the second.
Find var1=
Replace the var1 var2 assignment lines with
df -h|awk '{printf "%10s\t",$6 }'
df -h|awk '{print "%10s\t",$5 }'
The difference is that when you assign output to a variable which was mentioned earlier by other, the data loses formatting and is combined..
Find var1=
Replace the var1 var2 assignment lines with
df -h|awk '{printf "%10s\t",$6 }'
df -h|awk '{print "%10s\t",$5 }'
The difference is that when you assign output to a variable which was mentioned earlier by other, the data loses formatting and is combined..
Arnold, we need to see the *raw* df output first, and we know that $5 and $6 are likely to be wrong.
Shweta, please run my last script, exactly as it is, and show us what "df -h" writes out.
Shweta, please run my last script, exactly as it is, and show us what "df -h" writes out.
Simon, $5 and $6 are not wrong, the issue is that as I think you noted the formating goes away when you assign the data to a variable.
var1=$(output from anything) will be seen as one long line separated by whitespaces.
since there is nothing done with the data within var1 there is no reason to assign and then echo out when one could just let the command df -h | awk .....
My last script included the various tests to account for shifting when the block device is an LVM that forces the usage information onto the second time, but since the asker instead of using the script as posted and testing it, keeps .... on responding with no this did not work, without providing what the new script is.
https://www.experts-exchange.com/questions/28991566/Fetching-data-from-Multiple-Linux-Server-using-Shell-Scripting.html?anchor=a41960714¬ificationFollowed=182357375&anchorAnswerId=41947193#a41947193
I started with whatever was being used to adjust to what ...
var1=$(output from anything) will be seen as one long line separated by whitespaces.
since there is nothing done with the data within var1 there is no reason to assign and then echo out when one could just let the command df -h | awk .....
My last script included the various tests to account for shifting when the block device is an LVM that forces the usage information onto the second time, but since the asker instead of using the script as posted and testing it, keeps .... on responding with no this did not work, without providing what the new script is.
https://www.experts-exchange.com/questions/28991566/Fetching-data-from-Multiple-Linux-Server-using-Shell-Scripting.html?anchor=a41960714¬ificationFollowed=182357375&anchorAnswerId=41947193#a41947193
I started with whatever was being used to adjust to what ...
Hi Arnold. Not sure about the $5 and $6. If, as you say, the disk name is long, then the line with the data about that disk will be missing its first field, so all fields will be off by one. So that, with the "assigning to a variable", means that the data is all over the place.
We'll just have to wait to see what the bare "df -h" tells us!
We'll just have to wait to see what the bare "df -h" tells us!
simon, I ran the test
and since I have systems that use LVM filesystems of the type /dev/mapper/volumegroup-lo gicalvolum ename
filesystem size used avail Use% mounted on
/dev/mapper/volumegroup-lo gicalvolum ename pushes everything else onto the next line
size used avail use% mounted on
This is why I changed instead of using fixed fields to test whether the line has more than one record at which point to output either the last $(NF) mounted on or $(NF-1) for USE%
The asker was not responsive to what issues if any there were with that script nor what output was being received...
and since I have systems that use LVM filesystems of the type /dev/mapper/volumegroup-lo
filesystem size used avail Use% mounted on
/dev/mapper/volumegroup-lo
size used avail use% mounted on
This is why I changed instead of using fixed fields to test whether the line has more than one record at which point to output either the last $(NF) mounted on or $(NF-1) for USE%
The asker was not responsive to what issues if any there were with that script nor what output was being received...
ASKER
Hi Arnold, you can't say that it am not responsive. Everytime whenever you all asked for the output i posted that but due to inaccessibility to the server's i am not able to run the commands over there and enable to post the output here to figure it out and it doesn't mean that it am not responsive. If i was not responsive then never used to say that please do this ASAP.
The remote server you are running the script against can be the system you are on
It would go better without us having to ask for output or what you mean. If something is not working or matches, you kept simply answering no.
If you change or alter the script you are running, post the script and post the output within the "code" this way it will preserve the appearance you see versus adjusting the display here.
In the servers.txt have one IP, 127.0.0.1 and see when running the script connecting to ...
Do you get the output you want.
It would go better without us having to ask for output or what you mean. If something is not working or matches, you kept simply answering no.
If you change or alter the script you are running, post the script and post the output within the "code" this way it will preserve the appearance you see versus adjusting the display here.
In the servers.txt have one IP, 127.0.0.1 and see when running the script connecting to ...
Do you get the output you want.
Hi Shweta, what we need to get this right is the bare df output, then we can tell how comprehensive we need to be to get the formatting right. If, for example, none of the mountpoints have a space in them, then @arnold's $NF and $NF-1 wil be fine. If they do, we'll need my grep and sed lines.
If the machine that you are running the script on is hard to reach, we need to make the most of each visit!
If the machine that you are running the script on is hard to reach, we need to make the most of each visit!
ASKER
Thank you @Simon that command worked for me.
What information are you trying to get
ssh user@remoethost 'execute command' Any resulting output the data is sent back to the calling shell
if the executed command supresses any/all output (including error) that would explain why you see no data coming back.
If you add redirects at the end you must escape them if you want the redirects to work on the system where the shell script runs