?
Solved

Grep in a If Statement - Bash Shel

Posted on 2009-04-15
9
Medium Priority
?
3,006 Views
Last Modified: 2013-12-26
I've got a shell script, it looks for URLS in a file and does stuff. (See code below)

The if statement here

if [ -f "$CACHEFILE" -a -n $(grep -i $(basename $url) "$CACHEFILE") ] ; then

I thought would work, but it won't. If it finds the URL in the cachefile, it should stop the loops. Can anyone help?

cat <> $DLBASKET | (
        while read url ; do
                # First, check the cache if the file has been already downloaded
                if [ -f "$CACHEFILE" -a -n $(grep -i $(basename $url) "$CACHEFILE") ] ; then
                    echo "$DATETIME File exists in cache. Already downloaded - Skipping: $url" >> $LOGFILE
                else
                        echo "$DATETIME Starting with rate $LIMIT/s: $url" >> $LOGFILE
                        if [ $(expr match "$url" '[rapidshare.com]') = 1 ] ; then
                                # If it is a Rapidshare.com link, load the RS cookie
                                echo "RAPIDSHARE LINK"
                                wget -c  --directory-prefix=$DLDIR --load-cookies $RSCOOKIE $url
                        else
                                wget -c  --directory-prefix=$DLDIR $url
                        fi
                        echo "$DATETIME Finished: $url" >> $LOGFILE
                        echo $url >> $CACHEFILE
                fi
			done )
		 echo "All Files have been downloaded, clearing Download Basket"
		 rm $DLBASKET
		 touch $DLBASKET
		 echo "Finished!"
exit 0

Open in new window

0
Comment
Question by:jaxstorm
  • 3
  • 2
  • 2
  • +2
9 Comments
 
LVL 85

Expert Comment

by:ozo
ID: 24151467
-n "$(grep -i $(basename $url) $CACHEFILE)"
0
 
LVL 14

Expert Comment

by:Deepak Kosaraju
ID: 24151503
Replace the above if with following and give a shot...
if [ -f -a "$CACHEFILE" -n "$(grep -i $( basename $url) '$CACHEFILE')" ]

Open in new window

0
 
LVL 8

Author Comment

by:jaxstorm
ID: 24152510
Neither of those seems to work?
Any ideas?
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
LVL 85

Expert Comment

by:ozo
ID: 24152532
what did you try, and what was the result?
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24152628
I suspect that the url you are grepping for doesn't exist in the cachefile.  You need to verify your data.
0
 
LVL 14

Expert Comment

by:Deepak Kosaraju
ID: 24152636
Can you run the script in debug mode. using -x option

#sh -x scriptname

Open in new window

0
 
LVL 35

Accepted Solution

by:
Duncan Roe earned 2000 total points
ID: 24153155
I am a little suspicious of your test statement (i.e. [ ... ]). I think you can achieve what you want by using grep alone, something like:

if grep -i $(basename $url) "$CACHEFILE" >/dev/null 2>&1 ; then

This returns true if CACHEFILE exists and the URL is found in it, otherwise false.

You don't quote $CACHEFILE on line 16 - can it ever contain spaces or other special characters? If so, it'll never get created.

Using basename will only look at the remainder of the URL after the last / (see below) - is that what you really want?
07:54:10$ basename http://www.experts-exchange.com/OS/Linux/Q__24325376.html
Q__24325376.html

Open in new window

0
 
LVL 8

Author Comment

by:jaxstorm
ID: 24155000
Heres the situation as it stands. My university hosts a lot of content on rapidshare, and we all have a premium account. I'm trying to automate the downloading of the files using this shell script and wget. It works fine so far.

The files are passed from the DLBASKET into the CACHFILE so I know they've been downloaded. The script SHOULD check the CACHEFILE for the url and skip it if it exists there. It doesn't seem to be working.

In response to your comment duncan, I'm not too concerned about the use of basename, if it doesn't need to be in there then so beit, as long as the if statement works!

I've tried all the above posted answers, and each time it still downloads the file that it's the dlbasket, even though the url is in the cachefile.

Is there another way I can achieve this?
0
 
LVL 8

Author Closing Comment

by:jaxstorm
ID: 31570624
Turns out thise solution was the one I was looking for! Thanks for the help!
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Utilizing an array to gracefully append to a list of EmailAddresses
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Suggested Courses
Course of the Month17 days, 4 hours left to enroll

864 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question