Grep in a If Statement - Bash Shel

I've got a shell script, it looks for URLS in a file and does stuff. (See code below)

The if statement here

if [ -f "$CACHEFILE" -a -n $(grep -i $(basename $url) "$CACHEFILE") ] ; then

I thought would work, but it won't. If it finds the URL in the cachefile, it should stop the loops. Can anyone help?

cat <> $DLBASKET | (
        while read url ; do
                # First, check the cache if the file has been already downloaded
                if [ -f "$CACHEFILE" -a -n $(grep -i $(basename $url) "$CACHEFILE") ] ; then
                    echo "$DATETIME File exists in cache. Already downloaded - Skipping: $url" >> $LOGFILE
                else
                        echo "$DATETIME Starting with rate $LIMIT/s: $url" >> $LOGFILE
                        if [ $(expr match "$url" '[rapidshare.com]') = 1 ] ; then
                                # If it is a Rapidshare.com link, load the RS cookie
                                echo "RAPIDSHARE LINK"
                                wget -c  --directory-prefix=$DLDIR --load-cookies $RSCOOKIE $url
                        else
                                wget -c  --directory-prefix=$DLDIR $url
                        fi
                        echo "$DATETIME Finished: $url" >> $LOGFILE
                        echo $url >> $CACHEFILE
                fi
			done )
		 echo "All Files have been downloaded, clearing Download Basket"
		 rm $DLBASKET
		 touch $DLBASKET
		 echo "Finished!"
exit 0

Open in new window

LVL 8
jaxstormAsked:
Who is Participating?
 
Duncan RoeConnect With a Mentor Software DeveloperCommented:
I am a little suspicious of your test statement (i.e. [ ... ]). I think you can achieve what you want by using grep alone, something like:

if grep -i $(basename $url) "$CACHEFILE" >/dev/null 2>&1 ; then

This returns true if CACHEFILE exists and the URL is found in it, otherwise false.

You don't quote $CACHEFILE on line 16 - can it ever contain spaces or other special characters? If so, it'll never get created.

Using basename will only look at the remainder of the URL after the last / (see below) - is that what you really want?
07:54:10$ basename http://www.experts-exchange.com/OS/Linux/Q__24325376.html
Q__24325376.html

Open in new window

0
 
ozoCommented:
-n "$(grep -i $(basename $url) $CACHEFILE)"
0
 
Deepak KosarajuDevOps EngineerCommented:
Replace the above if with following and give a shot...
if [ -f -a "$CACHEFILE" -n "$(grep -i $( basename $url) '$CACHEFILE')" ]

Open in new window

0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

 
jaxstormAuthor Commented:
Neither of those seems to work?
Any ideas?
0
 
ozoCommented:
what did you try, and what was the result?
0
 
TintinCommented:
I suspect that the url you are grepping for doesn't exist in the cachefile.  You need to verify your data.
0
 
Deepak KosarajuDevOps EngineerCommented:
Can you run the script in debug mode. using -x option

#sh -x scriptname

Open in new window

0
 
jaxstormAuthor Commented:
Heres the situation as it stands. My university hosts a lot of content on rapidshare, and we all have a premium account. I'm trying to automate the downloading of the files using this shell script and wget. It works fine so far.

The files are passed from the DLBASKET into the CACHFILE so I know they've been downloaded. The script SHOULD check the CACHEFILE for the url and skip it if it exists there. It doesn't seem to be working.

In response to your comment duncan, I'm not too concerned about the use of basename, if it doesn't need to be in there then so beit, as long as the if statement works!

I've tried all the above posted answers, and each time it still downloads the file that it's the dlbasket, even though the url is in the cachefile.

Is there another way I can achieve this?
0
 
jaxstormAuthor Commented:
Turns out thise solution was the one I was looking for! Thanks for the help!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.