sandshakimi
asked on
Validate URLS?
www.centralasiacommerce.com
This is a business directory, listing links to various sites. Some links have been added a while back. Is there a tool that can scan the site and validate that all the URLs are still valid and work?
Plus: Can such tools verify that the company URL is unchanged? i.e. that if the company is doing a redirect to a new domain?
This is a business directory, listing links to various sites. Some links have been added a while back. Is there a tool that can scan the site and validate that all the URLs are still valid and work?
Plus: Can such tools verify that the company URL is unchanged? i.e. that if the company is doing a redirect to a new domain?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
All this is good feedback for me to
http://php.net/manual/en/filter.filters.validate.php
To scan the site, you might want to consider using a screen scraper. Any experienced PHP programmer can write one for you. I'll give it a try as time permits and post back here if I can get a good result. But I'm not optimistic. When I clicked the Tajikstan link in the header it seemed to take forever to get a response.
To detect what is changed and unchanged you need you have your own database of baseline and current URLs.
You may also want to apply some "human intelligence" to this project. For example, see http://www.famfamfam.com/ which is listed under web development. The link works, but the site itself appears to have entered the steady state in 2006. A lot has changed since then!