Link to home
Start Free TrialLog in
Avatar of firepol
firepol

asked on

PHP script that monitors a list of URLs and notify me if a page has been updated

Hi guys, does it exist a PHP script that will notify me if a URL has changed (updated, modified) like the following:

given a list of links of websites I'm interested to follow and see the changes/updates, in example:

http://community.vugames.com/WebX?14@90.O2jSemUSn9j.10434099@.1dd9900e

http://community.vugames.com/WebX?14@90.O2jSemUSn9j.10433936@.1dd98641

http://community.vugames.com/WebX?11@90.O2jSemUSn9j.10435135@.1dd962aa

The list could be in a .txt file, or in a simple mysql db...

The script should to the following

1) list these links, so that I can click them

2) download the links (like if you open the link with your browser and "Save Page As..." > "Web Page, HTML only" but the PHP script should save it)

3) modify the filename, including e.g. the date and time in it. In example it the link is

http://community.vugames.com/WebX?11@90.O2jSemUSn9j.10435135@.1dd962aa

it could save it as community.vugames.com_WebX?11@90.O2jSemUSn9j.10435135@.1dd962aa_2005.04.16_16.32.45
(the link name should be converted in a way that it works under linux, so the slashes could be converted in underscores and the “http:” could be removed…)

4) compare the last saved version with the version saved before (that's why having a date and time in the file is useful)

I don't know the perfect way of comparing to be able to say if a URL has been updated or not, but if it's a site without banners (or at least with a fixed tag like banner.gif, where the banner.gif rotates, but the html looks always the same) i think that comparing the file size is enough...

What I want to achieve is to see in a forum (a forum that has no auto-notification via e-mail) if new replies have been added to the thread, or if they have been modified by the authors...

5) if the 2 pages are different, the script should notify me, e.g. including a warning or icon near the link

6) an ideal thing would be to have an "history" link and see when the pages have been updated, so that if i accidentally click "refresh" 2 times I don't lose this information

7) the perfect thing would be that if there are updates there will be an added link that if I click I see what are the changes (like the output of the “diff” command under linux, something simple should be enough)

Do you know if there is already such a script elsewhere in the open source world?

If not, can you script the simplest solution to do what I mentioned above?

Thanks a bunch, 500 points if you do this ;)

sincerely,

Paolo
Avatar of ldbkutty
ldbkutty
Flag of India image

There won't be any such script that matches all your requirements. You have to do of your own or hire a programmer.

>> If not, can you script the simplest solution to do what I mentioned above?
Sorry, We are neither interested nor allowed to do your homework at EE.
Avatar of firepol
firepol

ASKER

>>There won't be any such script that matches all your requirements. You have to do of your own or hire a programmer.

Ok... if no script like this exists I'll try to make my own.

>>Sorry, We are neither interested nor allowed to do your homework at EE.

I will post here some php code if i have some troubles doing some functions, maybe you would be interested to assist me in this case...
Avatar of firepol

ASKER

Ah by the way I know it's impossible to find a script that meets ALL the requirements, but if you know some similar scripts that do a download of a URL and do a comparison, that would be a good start.
maybe you will have to use PEAR package Net_Monitor to do that, it is very flexible:
http://pear.php.net/package/Net_Monitor/docs/0.2.0/li_Net_Monitor.html
also take a look on other monitor systems :
http://www.hotscripts.com/Detailed/38137.html
http://www.hotscripts.com/Detailed/37676.html
ASKER CERTIFIED SOLUTION
Avatar of Vampireofdarkness
Vampireofdarkness
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Oh, also, add a mail() function if the file has changed if you want to be alerted via email.

Make this a cronjob for ever 5 minutes or something if you want it to automatically check. Otherwise you can manually check.. great fun if there are many urls and/or you are on a slow connection.
Avatar of firepol

ASKER

its not necessary that i create a cron, and the scriopt should not be completely automatic...

it's ok that i will wisit the script.php simply when i want to see the updates, the script will check all the urls seimply when i run it, and he should check them only once...

your for() function seems an infinite loop that would not be very appreciated by the web server admin...

i'll try to see what i can do with your suggested code...
It was left blank on purpose.
You need to make a script open the .txt containing urls, find how many there are and make this $urlcount, for example.
Then use: for($i=0; $i<$urlcount; $i++) {
No comment has been added to this question in more than 21 days, so it is now classified as abandoned..
I will leave the following recommendation for this question in the Cleanup topic area:
Accept: Vampireofdarkness {http:#13827344}

Any objections should be posted here in the next 4 days. After that time, the question will be closed.

Huji
EE Cleanup Volunteer