Solved

php sleep until file has been fully written

Posted on 2014-04-13
8
2,464 Views
Last Modified: 2016-09-14
how do you wait until a file has been fully written by an external process in php?

i'm trying to use this
function sleepUntilWritten($filename){
	while(true){
		$filesize_old = filesize($filename);
		sleep(4);
		$filesize_new = filesize($filename);
		if($filesize_old ==$filesize_new){
			echo 'done writing<br>';
			echo 'new = '.$filesize_new;
			echo 'old = '.$filesize_old;
			break;
			
		}
	}
}

Open in new window


but it breaks before the file has been fully written. i've also tried this:

function waitUntilFileWritten($filename){
	while(filectime($filename) >= time() - 2 ){
		sleep(1);
	}
}

Open in new window


but neither are getting the job done. i'm guessing the problem is more fundamental.
0
Comment
Question by:ChipmunkRumbleStud
  • 2
  • 2
  • 2
  • +2
8 Comments
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 39997572
The only way that the operating system knows that a program is done writing to a file is when that program closes the file handle.  I don't think PHP can 'see' that a file handle is open or closed.
0
 
LVL 34

Accepted Solution

by:
gr8gonzo earned 500 total points
ID: 39999020
Dave is partially correct - you'll have a hard time finding a way to query whether or not a file is opened by a separate application, but you can use flock() with LOCK_EX to try and obtain an exclusive writing lock on the file. If another app has the file open, then this would likely fail:

function check_is_still_writing($file)
{
  $is_still_writing = true;
  if($fp = fopen($file, "r+"))
  {
    if (flock($fp, LOCK_EX)) // Try to get an exclusive lock on the file
    {  
      flock($fp, LOCK_UN); // Unlock - we're just checking
      $is_still_writing = false;
    }
    fclose($fp);
  }
  return $is_still_writing;
}

Open in new window


The only problem with this is that if you have an application that does not keep the file handle open while writing, you could get a false positive. For example:

<?php
file_put_contents("file.txt", "blah blah");
sleep(10);
file_put_contents("file.txt", "blah blah", FILE_APPEND);
sleep(10);
file_put_contents("file.txt", "blah blah", FILE_APPEND);
?>

Open in new window


Each file_put_contents call opens the file handle, writes the data, and closes it. So the above script would take 20 seconds before it writes the final time, but it only has the file handle open for a few milliseconds in order to write out the blah blahs.

This can be common with logging, and because of that methodology, it becomes impossible to know when the writing has been finished because each "entry" is technically a completed/finished write. In that situation, you can do one of two things:

1. Use a much longer sleep() value to check the filesize differences (10-20 seconds instead of 4). Depending on what's writing to the file, there's usually some good number for a delay. Make SURE you use clearstatcache() to clear the filesystem info cache before checking the filesize again, since filesize is a cached function call:

http://www.php.net/manual/en/function.clearstatcache.php

It's possible even the four-second sleep() you had in your initial code is valid but the second filesize() call used the cached information because you didn't clear the filesystem info cache.

2. If you're exclusively on Linux, you can use shell_exec to run lsof on the various process IDs to try and discover what files they have open and try to find the one that is writing to that file, and then sleep until the process ends. Again, if the app opens/closes the file handle, this could be tricky to find the right process. If this PHP code is exceuted via a web server, you may need sudo to run lsof with the proper privileges.
0
 
LVL 109

Expert Comment

by:Ray Paseur
ID: 40003860
PHP is not designed for monitoring an external process.  You would want the external process to monitor itself and send a signal to a PHP script to tell it when the external process completes.  You might follow the example of PayPal Instant Payment Notification, which uses a POST-method request to start an "IPN" script on your server whenever a payment completes.  The IPN script has access to both the payment information and the information on your server, and it can be used to make data base entries, start other processes, etc.

If you're new to PHP and want to get a good foundation, this article can help lead you in the right directions (and away from the wrong directions).
http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/A_11769-And-by-the-way-I-am-new-to-PHP.html
0
Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

 

Author Comment

by:ChipmunkRumbleStud
ID: 40008294
@Ray Paseur so when you say 'sends a signal to php', what kind of signal is that and how do i implement it? I am salling the sound exchange binary to concatenate wave files and I want to make sure they are concatenated and the files are done before sending them to the client. If I enter

exec(sox one.wav two.wav oneandtwo.wav);
fopen(oneandtwo.wav);

Open in new window


will it work? The reason why I'm not testing it is because I'm having trouble calling binaries from php, even though 'dirs' works in the command line
0
 
LVL 109

Expert Comment

by:Ray Paseur
ID: 40008581
In computer science, "external process" is a term of art.  Now that I see your code example, I see that you've got an inline process using exec().  I think you probably want to try some tests just as you have written above, and experiment some.  I have always deliberately avoided exec() and I'm not conversant on how sox works.  What you'll be looking for is some affirmative signal that sox is finished and has created the file.  It may be as simple as trying to open the file, or it may take a little more work.

I'll sign off now, best of luck with your project, ~Ray
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 40008735
If you're running PHP from the web, then PHP ends up inheriting the often-LOWER privileges that the web server uses. So PHP ends up getting "crippled" and cannot access many binaries with exec/shell_exec/etc... The only way around this is to either elevate Apache's own permissions (not usually a good idea), or use sudo. There are a lot of guides on the internet on how to appropriately install a sudo plugin for your web server. Assuming your web server is Apache, just google for "php apache sudo" (without the quotes) and you should find at least one resource that matches your setup.

That should allow you to run sox from PHP, and assuming it runs synchronously (meaning that it will run until it finishes), then that's all you need. If it runs asynchronously (meaning it starts and then indicates that it's finished so that PHP moves on in the code but a background process continues to run to do the work), then you'll need to monitor the file.

As far as monitoring the file goes, please try the example I provided using flock.
0
 

Author Closing Comment

by:ChipmunkRumbleStud
ID: 40033387
I was actually trying to use a file that had been created by and external binary called by php's exec function. The php script waits until the process is finished though, so this ended up not being a problem
0
 

Expert Comment

by:Aaron Lowe
ID: 41798323
You could store the date/time the file was last modified, do the operation, then sleep in a loop until that date/time changes.
Then you know the file has been changed and is fully closed and ready for reading.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Generating table dynamically is the most common issue faced by php developers.... So it seems there is a need of an article that explains the basic concept of generating tables dynamically. It just requires a basic knowledge of html and little maths…
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question