Link to home
Start Free TrialLog in
Avatar of sai niharika sariputi
sai niharika sariputi

asked on

problem in while loop

i am trying to  using  if condition in while loop .but  loop is coming out after one iteration .
In file_names.txt contain text file names.after getting the names i am using ssh to connect another server and get the count of that file which is present that server.
this if condition is working fine when tried individually by hard coding the name.  

cat /usr/local/apps/jethro/devl/bin/file_names.txt|while read -r line;
do
	echo $line
	
	if [[ ${line} == DSI_BC_DIM* ]]; 
		then
			echo ${line}
			echo "DIM"
			b=`ssh  amzphxjethro05 'zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/'"${line}"' | wc -l'`
			echo ${b}
			echo `expr "${b}" - 1`
	else 
			echo "working"
			echo ${line}
			b=`ssh  amzphxjethro05 'zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/'"${line}"' | wc -l'`
			echo $b
	fi 
done

Open in new window

Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

The problem is that ssh reads standard input when it runs, so takes all of the "while read" loop's input lines.

Add the "-n" option to stdin, to stop it reading all standard input.
From my reading of the ssh man page, I wonder whether -n is really what you want.
If it works for you that's great; otherwise you could have better luck using a here document.
I experimented with cat, which also reads stdin if you don't give it any arguments
$ (cat <<////
> abcd
> ////
> )|rev
dcba
$ 

Open in new window

You have to run ssh (instead of cat) in a subshell (enclose in parentheses), because the document terminator has to appear on a line by itself (you can't put any other character after it). You can see that the pipe worked because rev has reversed the entered input. Here's how I suggest your line should look
b=`(ssh amzphxjethro05 <<//// 
zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/${line} |wc -l
////
)`

Open in new window

You want to put a matching backtick after the terminator, so I think you need to run in a subshell. You no longer need all the quoting you used to have though.
Hi Duncan, I would normally suggest adding
    </dev/null
to the command within a "while" loop, but ssh supports "-n" for that very purpose, which is neater.
Avatar of skullnobrains
skullnobrains

Or rather

whatever =`ssh -n host sh -c "\"zcat ...| wc \""`

... so the wc is run on the remote host. Otherwise you are sending the whole output of zcat over the network

Similarly

X=`ssh hostname sh -s`<<EOF
  zcat whatever | wc whatever
EOF

Should do the trick

I believe the latter is easier to read
@sai niharika sariputi

1. I used following code as per first comment from simon3270 using -n option
Using $() instead of ``
Using sed to remove all characters excluding number at Linux system here
#!/bin/bash
if [ -f /usr/local/apps/jethro/devl/bin/file_names.txt ]
then
        /usr/bin/cat /usr/local/apps/jethro/devl/bin/file_names.txt|while read -r line;
        do
                if [[ ${line} == DSI_BC_DIM* ]];
                then
                        echo "DIM line: $line"
                        #Using /usr/bin/sed to remove all characters and assign only wc -l number alone to variable b
                        b=$(echo -e $(/usr/bin/ssh -n amzphxjethro05 'zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/'"${line}"' | /usr/bin/wc -l') | /usr/bin/sed "s/.*\a//;")
                        echo "b: ${b} "
                        /usr/bin/expr "${b}" - 1
                        echo "-----------------"
                else
                        echo "working line: ${line}"
                        b=$(/usr/bin/ssh -n amzphxjethro05 'zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/'"${line}"' | /usr/bin/wc -l')
                        echo "b: ${b} "
                        echo "-----------------"
                fi
        done
else
        echo "/usr/local/apps/jethro/devl/bin/file_names.txt file not found"
fi

Open in new window


ONE MORE WAY:
2. Without using -n option using for loop
Using $() instead of ``
Using sed to remove all characters excluding number at Linux system here
#!/bin/bash
if [ -f /usr/local/apps/jethro/devl/bin/file_names.txt ]
then
        for LINE in $(/usr/bin/cat /usr/local/apps/jethro/devl/bin/file_names.txt)
        do
                if [[ ${LINE} == DSI_BC_DIM* ]];
                then
                        echo "DIM LINE: $LINE"
                        #Using /usr/bin/sed to remove all characters and assign only wc -l number alone to variable b
                        b=$(echo -e $(/usr/bin/ssh amzphxjethro05 'zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/'"${LINE}"' | /usr/bin/wc -l') | /usr/bin/sed "s/.*\a//;")
                        echo "b: ${b} "
                        /usr/bin/expr "${b}" - 1
                        echo "-----------------"
                else
                        echo "working LINE: ${LINE}"
                        b=$(/usr/bin/ssh amzphxjethro05 'zcat /md1/dev-scripts/PTD-Jethro-Stage/DSI_BC/'"${LINE}"' | /usr/bin/wc -l')
                        echo "b: ${b} "
                        echo "-----------------"
                fi
        done
else
        echo "/usr/local/apps/jethro/devl/bin/file_names.txt file not found"
fi

Open in new window

you are sending the whole output of zcat over the network and counting lines locally.
this is highly inefficient : you need to execute the wc remotely.

likewise, there is no point in spawning one ssh command per file
>> there is no point in spawning one ssh command per file
yes
we agree and it varies based on requirement, design document, ... :

for remoteHOSTIP in ...
do
...
for LINE in
...
.../usr/bin/ssh ... $remoteHOSTIP
done
...
done
yep. feel free to provide the corrected script. but you may notice the thread has been dead for about 6  months. i won't bother.
>> the thread has been dead for about 6  months
Thank you for the comments.
@sai niharika sariputi
if required, we will post updated script (based on your requirements and given files)
This question needs an answer!
Become an EE member today
7 DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.