Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

How to check for valid URLs?

Posted on 2003-11-11
4
Medium Priority
?
409 Views
Last Modified: 2008-03-17
Hi there,
      I am developing a site using PHP & MySQL. My DB contains thousands of URLs.

       For example:
            The URLs may be,

1, http://www.experts-exchange.com
2, http://www.experts-exchange.com/Web/Web_Languages/PHP
3, http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


      I am developing a page in that my client will click a button to get bad URLs i.e. the URLs that not open any page.

      For example:
            Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp


How do I check bad URLs without using fopen? Because, it takes more time to check valid URLs and causes time out error.

Kindly Can any one help me to solve this problem?

Thanks in advance.
0
Comment
Question by:DubsJoy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 14

Expert Comment

by:ThG
ID: 9723185
You may try with parse_url, and try to parse strings. But of corse you will find invalid urls, not non existent urls. What are you actually trying to do? Obviously, you can't distinguish between:
         Bad URL
 http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php

while the valid URL is

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp

without opening a connection to the remote server..
0
 
LVL 13

Accepted Solution

by:
lozloz earned 300 total points
ID: 9723319
hi,

if you use this script, it won't time out and will tell you which urls are responding and which aren't.. you'll need to leave it running for a while i suppose but it'll give you a correct answer. you can modify it to update the database or remove dead entries if you want

<?
$query = "SELECT * FROM urls"; # change this query and the index of $row to match the column name
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_assoc($result)) {
  set_time_limit(60);
  $url = @fopen($row["url"], "r");
  if($url) {
    print $row["url"] . " loads successfully<br />\n";
  } else {
    print $row["url"] . " failed to load..<br />\n";
  }
}
?>

tell me how you get on

loz
0
 

Author Comment

by:DubsJoy
ID: 9723371
Hi ThG,
    yeah with out opening a connection to the remote server it is not possible.
i need faster checking for valid URLs.

For example:

code should say
http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.php 
is bad URL (becouse this page is not exist) and

http://www.experts-exchange.com/Web/Web_Languages/PHP/askQuestion.jsp
is valid URL
0
 
LVL 14

Expert Comment

by:ThG
ID: 9724332
you have no faster ways than fopen(). Even if you write your own fsockopen(), request, and parse the output, you won't gain any appreciable speed..
0

Featured Post

New benefit for Premium Members - Upgrade now!

Ready to get started with anonymous questions today? It's easy! Learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I imagine that there are some, like me, who require a way of getting currency exchange rates for implementation in web project from time to time, so I thought I would share a solution that I have developed for this purpose. It turns out that Yaho…
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
The viewer will learn how to dynamically set the form action using jQuery.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

704 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question