Solved

How to check for valid URLs?

Posted on 2003-11-11
4
403 Views
Last Modified: 2008-03-17
Hi there,
      I am developing a site using PHP & MySQL. My DB contains thousands of URLs.

       For example:
            The URLs may be,

1, http://www.experts-exchange.com
2, http://www.experts-exchange.com/Web/Web_Languages/PHP
3, http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


      I am developing a page in that my client will click a button to get bad URLs i.e. the URLs that not open any page.

      For example:
            Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


How do I check bad URLs without using fopen? Because, it takes more time to check valid URLs and causes time out error.

Kindly Can any one help me to solve this problem?

Thanks in advance.
0
Comment
Question by:DubsJoy
  • 2
4 Comments
 
LVL 14

Expert Comment

by:ThG
ID: 9723185
You may try with parse_url, and try to parse strings. But of corse you will find invalid urls, not non existent urls. What are you actually trying to do? Obviously, you can't distinguish between:
         Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp

without opening a connection to the remote server..
0
 
LVL 13

Accepted Solution

by:
lozloz earned 100 total points
ID: 9723319
hi,

if you use this script, it won't time out and will tell you which urls are responding and which aren't.. you'll need to leave it running for a while i suppose but it'll give you a correct answer. you can modify it to update the database or remove dead entries if you want

<?
$query = "SELECT * FROM urls"; # change this query and the index of $row to match the column name
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_assoc($result)) {
  set_time_limit(60);
  $url = @fopen($row["url"], "r");
  if($url) {
    print $row["url"] . " loads successfully<br />\n";
  } else {
    print $row["url"] . " failed to load..<br />\n";
  }
}
?>

tell me how you get on

loz
0
 

Author Comment

by:DubsJoy
ID: 9723371
Hi ThG,
    yeah with out opening a connection to the remote server it is not possible.
i need faster checking for valid URLs.

For example:

code should say
http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php
is bad URL (becouse this page is not exist) and

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp
is valid URL
0
 
LVL 14

Expert Comment

by:ThG
ID: 9724332
you have no faster ways than fopen(). Even if you write your own fsockopen(), request, and parse the output, you won't gain any appreciable speed..
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

This article will explain how to display the first page of your Microsoft Word documents (e.g. .doc, .docx, etc...) as images in a web page programatically. I have scoured the web on a way to do this unsuccessfully. The goal is to produce something …
Things That Drive Us Nuts Have you noticed the use of the reCaptcha feature at EE and other web sites?  It wants you to read and retype something that looks like this.Insanity!  It's not EE's fault - that's just the way reCaptcha works.  But it is …
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now