Solved

How to check for valid URLs?

Posted on 2003-11-11
4
407 Views
Last Modified: 2008-03-17
Hi there,
      I am developing a site using PHP & MySQL. My DB contains thousands of URLs.

       For example:
            The URLs may be,

1, http://www.experts-exchange.com
2, http://www.experts-exchange.com/Web/Web_Languages/PHP
3, http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


      I am developing a page in that my client will click a button to get bad URLs i.e. the URLs that not open any page.

      For example:
            Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


How do I check bad URLs without using fopen? Because, it takes more time to check valid URLs and causes time out error.

Kindly Can any one help me to solve this problem?

Thanks in advance.
0
Comment
Question by:DubsJoy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 14

Expert Comment

by:ThG
ID: 9723185
You may try with parse_url, and try to parse strings. But of corse you will find invalid urls, not non existent urls. What are you actually trying to do? Obviously, you can't distinguish between:
         Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp

without opening a connection to the remote server..
0
 
LVL 13

Accepted Solution

by:
lozloz earned 100 total points
ID: 9723319
hi,

if you use this script, it won't time out and will tell you which urls are responding and which aren't.. you'll need to leave it running for a while i suppose but it'll give you a correct answer. you can modify it to update the database or remove dead entries if you want

<?
$query = "SELECT * FROM urls"; # change this query and the index of $row to match the column name
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_assoc($result)) {
  set_time_limit(60);
  $url = @fopen($row["url"], "r");
  if($url) {
    print $row["url"] . " loads successfully<br />\n";
  } else {
    print $row["url"] . " failed to load..<br />\n";
  }
}
?>

tell me how you get on

loz
0
 

Author Comment

by:DubsJoy
ID: 9723371
Hi ThG,
    yeah with out opening a connection to the remote server it is not possible.
i need faster checking for valid URLs.

For example:

code should say
http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php 
is bad URL (becouse this page is not exist) and

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp
is valid URL
0
 
LVL 14

Expert Comment

by:ThG
ID: 9724332
you have no faster ways than fopen(). Even if you write your own fsockopen(), request, and parse the output, you won't gain any appreciable speed..
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
PHP $_POST vs asp request 4 58
Dump data from mysql to xls php 10 56
Load string Array from file 23 42
How to convert PHP array into string ? 16 28
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
Many old projects have bad code, but the budget doesn't exist to rewrite the codebase. You can update this code to be safer by introducing contemporary input validation, sanitation, and safer database queries.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question