We help IT Professionals succeed at work.

Validating if a URL is valid (exists)

ParadyneDesigns
on
339 Views
Last Modified: 2012-08-13
Hello everyone

I know that you can valid a URL using an fopen statement but that is causing a problem.  I am not attempting to validate one URL, or two URL's but potentially thousands and thousands.  The fopen system is simply too slow!

I am currently using if(!fopen($domain, "r")) and then creating error code if it fails which I store and report to the user.  However, its simply not fast enough and is going to cause my script to potentially crash onto itself as it runs every 30 minutes (which would result in a never ending loop with the script kicking off again and again every 30 minutes).

The script in its current state can handle around 24,000 per hour but I need a lot more.  All it takes is one slow site with a lot of links and my script would crawl.

The fopen I believe is loading the complete page which is not necessary.  I just need a response back saying "yes, something there" or "no, not exist".  Is there anyway to validate like this.

My only alternative seems to be to not validate these links and all and hope that the users got it correct!!

Anyone got any ideas at all.

Cheers
Comment
Watch Question

Enterprise Web Developer
CERTIFIED EXPERT
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.