I know that you can valid a URL using an fopen statement but that is causing a problem. I am not attempting to validate one URL, or two URL's but potentially thousands and thousands. The fopen system is simply too slow!
I am currently using if(!fopen($domain, "r")) and then creating error code if it fails which I store and report to the user. However, its simply not fast enough and is going to cause my script to potentially crash onto itself as it runs every 30 minutes (which would result in a never ending loop with the script kicking off again and again every 30 minutes).
The script in its current state can handle around 24,000 per hour but I need a lot more. All it takes is one slow site with a lot of links and my script would crawl.
The fopen I believe is loading the complete page which is not necessary. I just need a response back saying "yes, something there" or "no, not exist". Is there anyway to validate like this.
My only alternative seems to be to not validate these links and all and hope that the users got it correct!!
Anyone got any ideas at all.