Link to home
Start Free TrialLog in
Avatar of sandshakimi
sandshakimiFlag for United States of America

asked on

Validate URLS?

www.centralasiacommerce.com

This is a business directory, listing links to various sites. Some links have been added a while back. Is there a tool that can scan the site and validate that all the URLs are still valid and work?

Plus: Can such tools verify that the company URL is unchanged?  i.e. that if the company is doing a redirect to a new domain?
SOLUTION
Avatar of Daniel Wilson
Daniel Wilson
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
This is a question with a lot of "latitude" in the answer.  Please see the PHP answer to FILTER_VALIDATE_URL on this page.
http://php.net/manual/en/filter.filters.validate.php

To scan the site, you might want to consider using a screen scraper.  Any experienced PHP programmer can write one for you.  I'll give it a try as time permits and post back here if I can get a good result.  But I'm not optimistic.  When I clicked the Tajikstan link in the header it seemed to take forever to get a response.

To detect what is changed and unchanged you need you have your own database of baseline and current URLs.

You may also want to apply some "human intelligence" to this project.  For example, see http://www.famfamfam.com/ which is listed under web development.  The link works, but the site itself appears to have entered the steady state in 2006.  A lot has changed since then!
SOLUTION
Avatar of Julian Hansen
Julian Hansen
Flag of South Africa image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of sandshakimi

ASKER

All this is good feedback for me to