Link to home
Start Free TrialLog in
Avatar of GAUTAM
GAUTAMFlag for United States of America

asked on

Help needed with bash shell script

Can anyone explain the bash script line by line.
scp -i ~/.ssh/id_rsa.sample gaara@stuid.student.com:ready/$2/*.zip ./$2 > slate.out 2>&1

ls -1 $2/* > curr.lst 2>/dev/null

while
read i
do
  if
    test -e ../done/"$i"
  then
      diff "$i" ../done/"$i" >/dev/null 2>&1
    if
      test $? -eq 0
    then
      rm "$i"
    fi
  fi
done < curr.lst

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of Member_2_6582184
Member_2_6582184
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of GAUTAM

ASKER

@helge000 : Thanks for the reply.

I got almost all of what you explained.

I have one query in the scp command :

# this line opens scp with a fixed key. It takes the second argument/parameter ($2) of the script to copy the Zip files so slate.out
scp -i ~/.ssh/id_rsa.sample gaara@stuid.student.com:ready/$2/*.zip ./$2 > slate.out 2>&1

Here is the zip file pulled from the server and placed in $2 directory right. And what is the purpose of slate.out here.
The slate.out is like a log file. It takes all output of the scp, redirects error and standard out into that file:
# redirects output into file  - but not only standard out but standard error out as well
> slate.out                     2>&1

Open in new window

Please keep my comment about the wildcard evail in mind. This script would only copy files which are in your current working dir on localhost and have a .zip extension and have the same name on remote, and would overwrite these. It is only seldomly wanted.
To further clarify: You would need a file called hello.zip in your local directory to get a file named hello.zip pulled from remote; the local file will then be overwritten.
I assume you rather want to copy all files with the extension *.zip on remote, then you would need to escape *:
scp -i ~/.ssh/id_rsa.sample 'gaara@stuid.student.com:ready/$2/\*.zip' ./$2 > slate.out 2>&1

Open in new window

If the difference is not obvious: 'user@server:/\*.zip'
Avatar of GAUTAM

ASKER

@helge000 : Thanks for replying.

So do you mean to say only if I have a file named hello.zip, only then scp would pull the file from the remote server to the local server or else discard the file.

So if I don't  have a file named sample.zip in my local then the server would not pull the file named sample.zip even if it is present in the remote server.

Can you elaborate why is this happening.
Correct.

You use a wildcard, in your case *. But that wildcard needs to be evaluated. In unix, this eval will happen at the first possible moment - eg. after you press enter. In your case, this would happen in the current working dir, because it matches, and it will match every file with a zip extension. Image it like this:
scp user@server:./(ls *.zip) .

Open in new window

which will result in the local matches:
$ ls .
. .. hello.zip sample.zip
$ ssh user@server ls . 
. .. hello.zip xyz.zip
$ scp user@server:./hello.zip .
$ scp user@server:./sample.zip .
scp: sample.zip: No such file or directory

Open in new window

With the escape char, you effectively delay the expansion/evaluation one step, since the escape char is 'removed' after the command was locally and passing it on to the character, not the wildcard, to the remote host. The remote host in turn can expand/evaluate the selection, since it 'sees' the wildcard special character.

But you can see, all this is far from transparent. Do yourself a favor and use rsync for that, try:
rsync -a --include '*.zip' --exclude '*' -i ~/.ssh/id_rsa.sample gaara@stuid.student.com:ready/$2/ ./$2

Open in new window

(Note: You need to explicitly exclude all stuff and then include your zips again)
Avatar of simon3270
The * in the scp line is fine - it will be evaluated on the remote host.

This is because the expansion is of the entire token - in this case:
    gaara@stuid.student.com:ready/$2/*.zip

So unless you have a directory called gaara@stuid.student.com:ready in your current directory (with a subdirectory called the value of $2, and .zip files in that subdirectory), then no files on the loacl machine will match that pattern, so the * will be left unchanged.

You can test this yourself:

If you run
    echo *.zip
in a directory with ZIP files, it will echo the names of those ZIP files on standard out.

if you run
    echo not_there_at_all*.zip
(assuming you have no zip files starting "not_there_at_all), it will echo the string "not_there_at_all*.zip", including the asterisk.

So, the shell only expands * and ? if it can find a matching file.

As  @helge000 suggests, though, if the bare file name is passed as a parameter (without the gaara@stuid.student.com prefix), you *do* need to escape asterisks and question marks.
The * in the scp line is fine - it will be evaluated on the remote host.
But - this only the case if you have no such directory locally. In my experience handling of wildcards in scp/ssh only works as expected (and therefore always) when escaped:
http://docstore.mik.ua/orelly/networking_2ndEd/ssh/ch07_05.htm

IMHO the whole ssh/wildcard thing is prone to errors and should be avoided, esp. in scripts; since we still do not know if this script shall run on BSD or Linux (and therefore technically even do not know if SSH1 is used) - you should use rsync,
The chances of having a directory called "gaara@stuid.student.com:ready" are pretty slim, but as you say, escaping is always a good idea.
Avatar of GAUTAM

ASKER

@helge000 & @simon3270 : Thanks for the info.

I have one doubt.

scp -i ~/.ssh/id_rsa.sample gaara@stuid.student.com:ready/$2/*.zip ./$2 > slate.out 2>&1

In the above line we are pulling from an assumed directory "ready"  shouldn't  the command be:

scp -i ~/.ssh/id_rsa.sample gaara@stuid.student.com:/ready/$2/*.zip ./$2 > slate.out 2>&1

Here shouldn't the ready be preceded by "/" how can it pull without having the fill path from root directory.

BTW this command is running on Linux.
If you don't specify an absolute path (I.e. starting with a /) for a remote file, scp starts from the user's home directory (in this case, gaara's home directory on stuid.student.com).