• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 434
  • Last Modified:

Cron Jobs To Create Pseudo-Static Pages?

I have a php page that is query intensive.  I would like to create a cron job that runs every 5 minutes or so that takes the PHP file as input, and outputs a static HTML file to decrease server load.  I have NO idea how to do this.
0
WarriorPoet42
Asked:
WarriorPoet42
  • 14
  • 9
  • 2
  • +1
4 Solutions
 
dr_dedoCommented:
first, set your cron job to fire every 5 min and run a php page called convert.php
5 * * * * /usr/local/bin/php /home/user/crons/convert.php >/dev/null

/usr/local/bin/php   is the path to php on your server (your ISP should have provided this info already)
/home/domain/convert.php    is the path to your php script (no need to have it in the html folder)
>/dev/null    means, don't print the results on the screen (shhhhhh)

now, to create the static page
0- delete the temp page if any
1- create a new temp static page,
2- delete the orginal page
3- replace the deleted orginal page with the temp page
4- delete the temp page
notice, the code here uses old fashion file management as it works with every version of PHP
---------------------------
<?php
$srcurl = "http://localhost/index.php";
$tempfilename = "tempindex.html";
$targetfilename = "index.html";
@unlink($tempfilename);
$dynpage = fopen($srcurl, 'r');
if (!$dynpage) exit();
$htmldata = fread($dynpage, 1024*1024);
fclose($dynpage);
$tempfile = fopen($tempfilename, 'w');
if (!$tempfile) exit();
fwrite($tempfile, $htmldata);
fclose($tempfile);
copy($tempfilename, $targetfilename);
unlink($tempfilename);
?>

0
 
AndyAelbrechtCommented:
to add to dr_dedo's excellent answer:

if your provider did not provide the location of the php binary file (/usr/local/bin/php) you could find yourself with a little problem; however this is easily fixed with changing the crontab line to:

5 * * * * wget http://my.website.com/crons/convert.php >/dev/null

  basically, this will download the script (which you put in a web-accessible directory now); download a PHP script is the same as executing it, and you do not need Execute access to the CLI PHP interface (which, as far as I know, isn't installed by many webhosts). The script itself would be the exact same as the one dr_dedo provided.

cheers,
Andy
0
 
WarriorPoet42Author Commented:
Would one of you mind giving me a line by line of what the script does?  Code is great, but I like to understand what I'm executing. :D
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
AndyAelbrechtCommented:
dr_dedos script is quite simple to explain:

<?php
$srcurl = "http://localhost/index.php"; //set filename of DYNAMIC page you want to convert to STATIC page
$tempfilename = "tempindex.html"; //temp filename
$targetfilename = "index.html"; //target STATIC page filename
@unlink($tempfilename); //remove temp file (if still there)
$dynpage = fopen($srcurl, 'r'); //open the dynamic page and put the $dynpage as filepointer
if (!$dynpage) exit(); //if this didn't work (opening the dynamic page), then exit this script
$htmldata = fread($dynpage, 1024*1024); //read the contents of the page into the $htmldata variable
fclose($dynpage); //close the filepointer pointing to the dynamic page
$tempfile = fopen($tempfilename, 'w'); //open the temporary file (write mode) and put $tempfile as filepointer
if (!$tempfile) exit(); //if opening this file didn't work, exit the script
fwrite($tempfile, $htmldata); //write the contents of $htmldata (filepointer to dynamic page) into the temp page
fclose($tempfile); //close the tempfile
copy($tempfilename, $targetfilename); //copy the tempfile to the targetfile (targetfile being the STATIC page you want to generate)
unlink($tempfilename); //remove the tempfile
?>
0
 
Richard QuadlingSenior Software DeverloperCommented:
Something to watch out for. If you script takes longer than 5 minutes to run, you could end up with mutliple copies of the script running.

Something that is worth adding to your script is a semaphore to NOT run if already running.

Personally, I would use a simple mkdir option.

MKDIR will either succeed or fail. Nothing in between.

so

<?php
$b_semaphore_created = mkdir('LOCK_' . basename($_SERVER['PHP_SELF'], '.php'));
if (True === $b_semaphore)
 {
 // You can run your script as the lock was created.
 ...
 // Remove the lock.
 rmdir('LOCK_' . basename($_SERVER['PHP_SELF'], '.php'));
 }
?>

0
 
WarriorPoet42Author Commented:
RQ: If the script takes longer that 5 minutes to run, I am in a world of hurt.  It only takes a second to run . . . but multiply that times hundreds of users, and you can see why I would rather statisize it.

AA: I have a question about this line: $htmldata = fread($dynpage, 1024*1024);

I assume as it reads it into the variable, it is parsing it through the PHP processor and outputting pure HTML into $html data.  What does the 1024*1024 option do?

Thanks
0
 
Richard QuadlingSenior Software DeverloperCommented:
I have some programs which take 20 or 30 minutes to run. Generating emails and such and updating a static page with the current results.

Why not build the caching into the page? So only 1 user gets the hit.

Saves having an additional script and the possibility of an issue if the cron fails to run.

e.g.

<?php
$s_filename = './saved.html'; // What shall we serve if we are not old enough?
$i_delay = 300; // How long between refreshes?

if (file_exists($s_filename) && (filemtime($s_filename) < (time() - $i_delay)))
 {
 readfile($s_filename);
 }
else
 {
 ob_start();
 // Run your code here as it it was generating a normal page.
 // ...

 // Grab the output buffer as we want to display it as well as save it.
 $s_output = ob_get_clean();

 // Output the buffer.
 echo $s_output;

 // Save the output buffer for later.
 file_put_contents($s_filename, $s_output);
 }
?>
0
 
dr_dedoCommented:
1024*1024 is how much data to read at a time, no need to change that

you know, you don't have to run that script every 5 min. follow me here
now you got that index page that is having a lot of hits, this page got some dynamic contnents that come from a db, right ??
if writes to the db is not that frequent (say articles or so), you can run the script only after some data is inserted into the db.
say in some admin page, you add an article that will show in index.php, then, after the insert SQL, you can call the vonverting script!!

so, if you update your database 10 times per day, the page will be updated 10 times per day, and will always have the new data. meanwhile, you'll reduce server overload greatly!!
hope you got what i mean
0
 
Richard QuadlingSenior Software DeverloperCommented:
Just rereading the line ...

if (file_exists($s_filename) && (filemtime($s_filename) < (time() - $i_delay)))

I'm not sure I've got it right.

filemtime has to be within the last 300 seconds.

So, filemtime should be >= time - 300.

Doh!

if (file_exists($s_filename) && (filemtime($s_filename) >= (time() - $i_delay)))

filemtime say at 10:30
current time say is 10:31

filemtime >= (10:31 - 0:05)
 so serve.

Sorry about that!
0
 
Richard QuadlingSenior Software DeverloperCommented:
Good point dr_dedo.

If you are not inserting the data yourself (or by something you have no control over), is there a simple query you can do to determine if the data is old. Like the most recent id on a specific table, or the date on a table.

Say there is and the column is called last_entry.

<?php
// If the last_entry file exists then include it.
if (file_exists('./last_entry.inc'))
 {
 require_once ('./last_entry.inc');
 }

// If there is no last entry variable, define it.
if (!isset($last_entry))
 {
 $last_entry = 0;
 }

// Get the last entry indicator from the DB.
$s_SQL = 'SELECT TOP 1 last_entry FROM some_table ORDER BY last_entry DESC';
$r_conn = xxx_connect($server, $username, $password);
$r_results = xxx_query($s_sql, $r_conn);
$a_row = xxx_fetch_array($r_results);
xxx_free_results($r_results);

// Is the DB' s last_entry the same as our $last_entry?
if ($last_entry !== $a_row['last_entry'])
 {
 // Run your code to generate the page and use the output buffering mechanism I described earlier.

 // Save the last entry.
 file_put_contents('./last_entry.inc','<?php $last_entry = ' . $a_row['last_entry'] . '; ?>');
 }
?>
0
 
Richard QuadlingSenior Software DeverloperCommented:
Oh. If the last last entry is the same as the DBs, then output the cached file.
0
 
Richard QuadlingSenior Software DeverloperCommented:
So, 4 good ways.

1 - Use cron
2 - Use code on page to supply a cached copy based upon a delay period.
3 - Use code on page to detect a "stale" condition and use a cached copy.
4 - Create a cached copy when the data is updated.

0
 
WarriorPoet42Author Commented:
I really have no idea of when the most recent change would be - this table is a list of all users and some public data about them - data which not only can be changed at any time, but often is.  Sometimes dozen changes by a single user in a single session.
0
 
Richard QuadlingSenior Software DeverloperCommented:
Ok.

I would combine 2 of the above ways.

Create the static file when the data is changed, but only if the last change was greater then 5 minutes ago.

You can do all of that as part of the update.
0
 
Richard QuadlingSenior Software DeverloperCommented:
You HAVE to service the updates, so not a lot of choice there.

You only have to generate the output file once every file minutes at WORSE.

You may want to add a small message saying "This page is based on changes upto xxxx", where xxxx is the time of the last change.
0
 
WarriorPoet42Author Commented:
I will likey use option 2 - use code on page to supply a cached copy based on a delay period.

My code would likely look like this:
<?php
// This file: players.php
$s_filename = './static_players.txt'; // What shall we serve if we are not old enough?
$i_delay = 300; // How long between refreshes?
if (file_exists($s_filename) && (filemtime($s_filename) >= (time() - $i_delay)))
 {
 readfile($s_filename);
 }
else
 {
 ob_start();
 // The original contents of the players.php

 // Grab the output buffer as we want to display it as well as save it.
 $s_output = ob_get_clean();

 // Output the buffer.
 echo $s_output;

 // Save the output buffer for later.
 file_put_contents($s_filename, $s_output);
 }
?>

If that looks correct, I'll close the question.

If that looks correct
0
 
WarriorPoet42Author Commented:
The users can always see updated versions of their own data - this page is basically for competative purposes only.  So, based on the above code, I think I will go with serving a static page that is updated every five minutes - period.  I'll inlcude a note that data is updated every five minutes.  Or maybe even the time of the next update.
0
 
Richard QuadlingSenior Software DeverloperCommented:
There needs to be some code following

// The original contents of the players.php




This is where you would issue your queries and generate the HTML as normal.

Other this would work.
0
 
Richard QuadlingSenior Software DeverloperCommented:
Yes. Good idea. If you wanted to, you could do clever things like having team leaders force the latest set.

If the logged in user is a team leader within the game (if such a concept applies), then set the $i_delay to 0.

This will mean that they always get the true results and not a cached copy.

It would be worth tracking how much caching you are doing and comparing it with the amount of time it takes to actually run the queries.

0
 
WarriorPoet42Author Commented:
RQ: That last suggestion is spot on.

And yes, where I put "// The original contents of the players.php" is where, quite literally, I would put the full code of my current (non-cached) version of that file, correct?
0
 
Richard QuadlingSenior Software DeverloperCommented:
More or less.

Remove leading <?php and trailing ?>

And that should be it.

One down side is any errors you have will sit in the file for 5 minutes. If you have errors, I would handle them in code properly rather than letting PHP just fall out with them.
0
 
WarriorPoet42Author Commented:
Oh damn!  I just thought of something.  The table in the page have options for sorting - currently selecting a new sort reloads the page with the optional sorts added to the queries.  Clearly, dynamic sorting would not work on a cached page . . .

Any ideas? :/
0
 
Richard QuadlingSenior Software DeverloperCommented:
Why not? Instead of just caching the HTML, cache the data also. In arrays and then you can sort the arrays as you want to display the data.

The PAGE is not the issue then, but the data.

0
 
Richard QuadlingSenior Software DeverloperCommented:
Home now. Back tomorrow to explain how to do this.

But 3 files.

1 - to generate the data file.
2 - the data file.
3 - the script which the user uses to sort the data as they like.
0
 
WarriorPoet42Author Commented:
I'm glad you'll be back soon.  In the mean time, I'll close this question - as it has been clearly and completely answered - and open a new question about dynamically sorting cached data.
0
 
WarriorPoet42Author Commented:
0
 
Richard QuadlingSenior Software DeverloperCommented:
Thanks for the points. More comments on your new question. Interesting question. Something I've done a few times myself, but never placed in concrete.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

  • 14
  • 9
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now