Solved

Code to check text on a website

Posted on 2012-03-27
2
323 Views
Last Modified: 2012-03-28
I'm trying to write some code that will check regularly (say every few minutes) whether a particular user is online on a website. When that person comes online or goes offline I need it to send me an email. I could run it on a variety of platforms - ideally it would sit on a box that runs Debian Squeeze (6) but if OS X or Windows would be easier then that could work too.

What I'm looking for with this question is just some pointers on the best way to get started with this. I'm not expecting a turnkey solution and am more than prepared to do the learning and put the hard work in myself. Having said that I'm reasonably technical but my Linux skills are only very moderate - I've never written a script for example.

The basic structure is, I think, pretty clear. There needs to be code that checks the website for specific text that matches the name of the user that I'm interested in. When it finds or doesn't find the text it checks against the last known status and if that has changed it sends the email. Delay a while then repeat.

For the website checking I've been experimenting with:

      curl <url> | awk '/<matchtext>/ {<action>}'

That sort of works with some sites. But the specific problem is that the particular website I'm interested in is session based (not sure if I have the terminology right here) in that when I open the site in a browser it logs me in (as an anonymous/guest user - no username or password required) and the page just shows "please wait while we log you on" (or something like that) for 10-15 seconds before the page that I need to search on loads. How can I create that session using code?

As I said at the start I'm just looking for pointers on how to think about this. Really grateful for any help.

Thanks, Chris
0
Comment
Question by:chrwil
2 Comments
 
LVL 11

Accepted Solution

by:
un1x86 earned 500 total points
ID: 37775602
Hi Chris

Not sure if I understood you right. But you could use lynx. If you need to login you could record the steps with

lynx -cmd_log=filename http://theurl.com

and then run the script with

lynx -cmd_script=filename http://theurl.com

cmd_log will log all steps you do on lynx. With cmd_script it will run automated. You could do it as following

use cmd_log script
navigate to login
login
navigate to the webpage where you can see the logged in user
press "p"
enter path where to save page
exit

Then you run
lynx -cmd_script=filename http://theurl.com

this will save the page to the path you have given. You can proceed with that file by grepping its content.

Hope that helps. There is some good documentation out there using cmd_script.
http://blog.unixy.net/2009/06/script-to-automate-browsing-actions-using-lynx/

CURL by the way can do quite a lot with sessions. But you need to know how the login process works on the website. You can send POST variables using CURL and save the generated session.

Just google for CURL post session
0
 

Author Comment

by:chrwil
ID: 37780187
Thanks. I've been playing with Lynx since I saw your post. It may be the answer.
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Although it can be difficult to imagine, someday your child will have a career of his or her own. He or she will likely start a family, buy a home and start having their own children. So, while being a kid is still extremely important, it’s also …
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question