?
Solved

Code to check text on a website

Posted on 2012-03-27
2
Medium Priority
?
352 Views
Last Modified: 2012-03-28
I'm trying to write some code that will check regularly (say every few minutes) whether a particular user is online on a website. When that person comes online or goes offline I need it to send me an email. I could run it on a variety of platforms - ideally it would sit on a box that runs Debian Squeeze (6) but if OS X or Windows would be easier then that could work too.

What I'm looking for with this question is just some pointers on the best way to get started with this. I'm not expecting a turnkey solution and am more than prepared to do the learning and put the hard work in myself. Having said that I'm reasonably technical but my Linux skills are only very moderate - I've never written a script for example.

The basic structure is, I think, pretty clear. There needs to be code that checks the website for specific text that matches the name of the user that I'm interested in. When it finds or doesn't find the text it checks against the last known status and if that has changed it sends the email. Delay a while then repeat.

For the website checking I've been experimenting with:

      curl <url> | awk '/<matchtext>/ {<action>}'

That sort of works with some sites. But the specific problem is that the particular website I'm interested in is session based (not sure if I have the terminology right here) in that when I open the site in a browser it logs me in (as an anonymous/guest user - no username or password required) and the page just shows "please wait while we log you on" (or something like that) for 10-15 seconds before the page that I need to search on loads. How can I create that session using code?

As I said at the start I'm just looking for pointers on how to think about this. Really grateful for any help.

Thanks, Chris
0
Comment
Question by:chrwil
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 11

Accepted Solution

by:
Chris Sandrini earned 2000 total points
ID: 37775602
Hi Chris

Not sure if I understood you right. But you could use lynx. If you need to login you could record the steps with

lynx -cmd_log=filename http://theurl.com

and then run the script with

lynx -cmd_script=filename http://theurl.com

cmd_log will log all steps you do on lynx. With cmd_script it will run automated. You could do it as following

use cmd_log script
navigate to login
login
navigate to the webpage where you can see the logged in user
press "p"
enter path where to save page
exit

Then you run
lynx -cmd_script=filename http://theurl.com

this will save the page to the path you have given. You can proceed with that file by grepping its content.

Hope that helps. There is some good documentation out there using cmd_script.
http://blog.unixy.net/2009/06/script-to-automate-browsing-actions-using-lynx/

CURL by the way can do quite a lot with sessions. But you need to know how the login process works on the website. You can send POST variables using CURL and save the generated session.

Just google for CURL post session
0
 

Author Comment

by:chrwil
ID: 37780187
Thanks. I've been playing with Lynx since I saw your post. It may be the answer.
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Today, the web development industry is booming, and many people consider it to be their vocation. The question you may be asking yourself is – how do I become a web developer?
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…
Introduction to Processes
Suggested Courses

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question