We help IT Professionals succeed at work.

LoadHTMLFile() loading time in PHP

duncanb7
duncanb7 asked
on
2,064 Views
Last Modified: 2012-06-27
Dear Expert,

in my following php code I do automation for grab data from 10 webpage  at the same website,
 sometimes, I get hangup and I guess the reason url link is busy so let loadHTMLfile() keep waiting.
it takes for a really long ocassionlly such as  15-30 minutes. the webpage size is really small as 30k.
Why my php code of loadHTMLFile()  is no timing limit and fall back to next code once the time waiting
is too long.
Do you have any method to timer the waiting time in php code, once the time is expired  and then
go to next  line of code in same php program ?
In VBA I will use this but the code in the loop is never in idle and never  waiting forever so it will be okay
but for php's loadHTMLFile() that won't work if using similar while loop . Any suggestion, please advise

VBA code to set timer for waiting
=======================
a=Time()
Do until   TimeValue(Time()) - TimeValue(a) > TimeValue("00:01:00")
'code here
Loop




<?php
For ($k=0; $k <4; ++$k)  {  //repeat loadhtmlfile() over again if one of $c is not loaded
try {
for ($c=0; $c < 10; ++$c) {
$url= 'http://www.othersite.com/ex.aspx?symbol='.$c;
$dom = new DOMDocument();
$dom->loadHTMLFile($url);
echo "start=".$c;
$data= $dom->getElementsByTagName('table')->item(2)>nodeValue;
echo "Sucess to pass getElementsByTagname".$c;
}
$k=4;//It means no need to do re-do since no fatal error during loadHTMlfile for 10 pages
}
catch(err) {
echo "It found error at =" %c;
}
}
?>

Open in new window

Comment
Watch Question

Most Valuable Expert 2011
Author of the Year 2014
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION

Author

Commented:
No never

How to ue the function, where I should  put ?
Where I can set the timeout time for example, 60s only? Where $timeout=3, means 3 minuts ?

Author

Commented:
You mean use curl to get html file instead of loadHTMLfile(), Right ?
Most Valuable Expert 2011
Author of the Year 2014

Commented:
Yes, you would use the function to get the HTML file into a string variable in your script.  Then you could store the string on your own server and process it there.  There would be no further delay because of HTTP or remote server issues.

The script I posted above has an example of the use case (actually 2 examples).  These start on line 85.

Author

Commented:
Coclusion:
1-Timeout is working fine and error message coming once timeout
2- $arg arrary is on fire but on always on wrong webpage probably it is casued by url code issue
for my case I don't know why, so  and I need to set as follows using http_build_query()
$url = array('Symbol'=> $c);
my_curl("http://www...../test.aspx?&".http_build_query($url),5,True);

and then it works fine exactly,
3- The speed excution time is 40% faster using curl methond than loadHTMLFile() by estimsation.

Duncan


Taking out $arg arrary code and function input
=====================================
function my_curl
( $url
/////////////////, $get_array=array()
, $timeout=3
, $error_report=TRUE
)

// PREPARE THE ARGUMENT STRING IF NEEDED
  //  $get_string = '';
   // foreach ($get_array as $key => $val)
   // {
     //   $get_string
       // = $get_string
        //. urlencode($key)
        //. '='
        //. urlencode($val)
        //. '&';
   // }
   // $get_string = rtrim($get_string, '&');
   // if (!empty($get_string)) $url .= '?' . $get_string;

Author

Commented:
Thanks for your reply.
The code is help a lot

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a sample view!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.