• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1082
  • Last Modified:

Shell Script To Monitor Webpage For specific word

I want to have a shell script monitor a webpage for the word "Offline" and have it e-mail me if the word appears on the page.  How can I do this? Thanks
0
newmacguy
Asked:
newmacguy
  • 7
  • 5
  • 3
  • +3
1 Solution
 
omarfaridCommented:
you may monitor it with wget (see man wget) to get the page and then use grep for that word and if it is there then use mail or mailx o send mail
0
 
newmacguyAuthor Commented:
Any chance you can provide some sample code? I'm not very good at shell scripting. Thanks
0
 
woolmilkporcCommented:
lynx might be easier:

lynx -dump http://my.url.com | grep -q "Offline" && echo "$(date) Offline Alert!" | mailx -s 'String "Offline" found!' newmacguy@domain.tld

wmp
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
newmacguyAuthor Commented:
I tried that but I get this result:
  Not Acceptable

   An appropriate representation of the requested resource / could not be
   found on this server.

   Additionally, a 404 Not Found error was encountered while trying to use
   an ErrorDocument to handle the request.

This command works on other websites, but not this one for some reason.
0
 
TobiasHolmCommented:
406 Error (Not Acceptable)

Are you seeing this error when trying to access a page? This is due to Apache mod_security that is turned on by default. While you can use the following to diagnose the problem (turning the filter off should resolve the issue):
SecFilterEngine off

It's important to leave the filter on as it helps prevent spam and injection attacks. What you need to do is figure out (ask your hosting provider if you need help) the phrases that are triggering the filter. Typically these will include phrases that include possible commands such as "ftp," "telnet," "ssh," etc. Once you know which ones are, you can modify the filter to permit the page that is not loading (scan your logs or Google sitemap for 406 errors to find all the pages experiencing this problem):

Ref: http://community.contractwebdevelopment.com/406-error-not-acceptable-an-appropriate-representation-of-the-requested-resource-could-not-be-found-on-this-server
0
 
newmacguyAuthor Commented:
I don't control that particular web server.  I'm trying the wget method as that seems to work.  How can I make it send the e-mail if the grep returns results from the wgetted file?
0
 
omarfaridCommented:
try this

wget http://domain.com/file.html
grep -w Offline file.html
if [ $? -eq 0 ]
then
      mailx -s "Offline found" user@yourdomain
fi
rm file.html
0
 
newmacguyAuthor Commented:
I'm getting an error on the if statement line.
0
 
newmacguyAuthor Commented:
line 3: [1: command not found
0
 
TobiasHolmCommented:
Try omarfarid's code in a bash-script. But I suppose you know how to make such script-file?

In case you don't:

Put the code in a file. Make the file executable with: chmod a+x yourfilename.sh

Then run the script with: ./yourfilename.sh
#!/bin/bash
wget http://domain.com/file.html
grep -w Offline file.html
if [ $? -eq 0 ]
then
      mailx -s "Offline found" user@yourdomain
fi
rm file.html

Open in new window

0
 
omarfaridCommented:
did you copy the script exact ? what shell are you using?
0
 
omarfaridCommented:
you need to leave space before $?
0
 
newmacguyAuthor Commented:
I copied it exactly. I think the problem is that there are some $ symbols in the grepped text
0
 
omarfaridCommented:
the error is related to space before $?
can you post the script you run?
0
 
gelonidaCommented:
The  '[' AND ']'  need to be separated by a space from preceeding and following words
as omarfarid mentioned


please note also, you have to remove file.html after having grepped for it
otherwise the next wget command would save the result as file.html.1

if you want to keep the html file, then yyou could add following command
between line 6 and 7 of TobiasHolm's script

cp file.html saved_file.html


0
 
egarciatCommented:
You may want to try:


#!/bin/bash

alert_string="offline"

alertme() {
mailx -s "Offline found" user@yourdomain
}

wget -t 1 -T 5 --quiet -O - "http://yourdomain.com" 2>/dev/null | grep -i "$alert_string" && alertme

Open in new window

0
 
egarciatCommented:
The "-t" argument to wget is the number of tries it does before exiting, if you do not set this param, I think wget tries several times (which may be desired), the "-T" is the time out time in secons, if you do not specify this and for some reason the page is not reachable, wget may block your script for some predefined default wich I think is 45 secs.

You can specify whathever you want in the function "alertme".. personally I use sendmail for such operations, but any other program may work..

alert() {
echo -e "Subject: Offline Alert " | sendmail -falert@yourdomain.com youruser@youremaildomain &
}
0
 
egarciatCommented:
I Apologize for posting again..

I forgot to tell you that altough not necesary, I would be a good idea to set the content type of the web page you are downloading to "text/plain", you may want to use an HTML META tag.
0
 
newmacguyAuthor Commented:
Sorry for being away so long. This appears to have fixed it though.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 7
  • 5
  • 3
  • +3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now