Solved

How to check for valid URLs?

Posted on 2003-11-11
4
404 Views
Last Modified: 2008-03-17
Hi there,
      I am developing a site using PHP & MySQL. My DB contains thousands of URLs.

       For example:
            The URLs may be,

1, http://www.experts-exchange.com
2, http://www.experts-exchange.com/Web/Web_Languages/PHP
3, http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


      I am developing a page in that my client will click a button to get bad URLs i.e. the URLs that not open any page.

      For example:
            Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


How do I check bad URLs without using fopen? Because, it takes more time to check valid URLs and causes time out error.

Kindly Can any one help me to solve this problem?

Thanks in advance.
0
Comment
Question by:DubsJoy
  • 2
4 Comments
 
LVL 14

Expert Comment

by:ThG
ID: 9723185
You may try with parse_url, and try to parse strings. But of corse you will find invalid urls, not non existent urls. What are you actually trying to do? Obviously, you can't distinguish between:
         Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp

without opening a connection to the remote server..
0
 
LVL 13

Accepted Solution

by:
lozloz earned 100 total points
ID: 9723319
hi,

if you use this script, it won't time out and will tell you which urls are responding and which aren't.. you'll need to leave it running for a while i suppose but it'll give you a correct answer. you can modify it to update the database or remove dead entries if you want

<?
$query = "SELECT * FROM urls"; # change this query and the index of $row to match the column name
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_assoc($result)) {
  set_time_limit(60);
  $url = @fopen($row["url"], "r");
  if($url) {
    print $row["url"] . " loads successfully<br />\n";
  } else {
    print $row["url"] . " failed to load..<br />\n";
  }
}
?>

tell me how you get on

loz
0
 

Author Comment

by:DubsJoy
ID: 9723371
Hi ThG,
    yeah with out opening a connection to the remote server it is not possible.
i need faster checking for valid URLs.

For example:

code should say
http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php 
is bad URL (becouse this page is not exist) and

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp
is valid URL
0
 
LVL 14

Expert Comment

by:ThG
ID: 9724332
you have no faster ways than fopen(). Even if you write your own fsockopen(), request, and parse the output, you won't gain any appreciable speed..
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Both Easy and Powerful How easy is PHP? http://lmgtfy.com?q=how+easy+is+php (http://lmgtfy.com?q=how+easy+is+php)  Very easy.  It has been described as "a programming language even my grandmother can use." How powerful is PHP?  http://en.wikiped…
Deprecated and Headed for the Dustbin By now, you have probably heard that some PHP features, while convenient, can also cause PHP security problems.  This article discusses one of those, called register_globals.  It is a thing you do not want.  …
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

943 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now