Link to home
Start Free TrialLog in
Avatar of psykomotor
psykomotor

asked on

PHP copy() from network share slow

I have a PHP cron script which will run on a server and transfer files from another, but I am finding the copy() function extremely slow for transferring via a network share.

The first server hosts a company intranet/management system, and the second server hosts a PBX which creates call recordings. The overall objective is to transfer call recordings from the PBX to the intranet server and match them up with customer records.

The recorded files range from almost zero to 20MB. To test transfer speed I've copied the files using the windows GUI and a solid 10MB/sec is obtained, transferring over 100MB in just over 10 seconds.

After going through all sorts of headaches getting the permissions sorted out, the script now runs and using RecursiveDirectoryIterator, can list all the files on the 'recordings' share within about a second (800+ files in multiple subdirectories at this time), so I've cracked that much.

However, once I come to copy() the files, the script can hang until it times out (30 seconds). The script doesn't need to transfer everything in one go, but the more it can do in 1 run the better.

I've tested with a single 400kb file and even that will still time out. It's intermittent, as around one in 10 attempts will go through lightning fast - in one particular example it copied 15-20MB in a few seconds.

Any advice on what things I could check to find the bottleneck would be much appreciated !

Useful details:

- Vista Business SP2 (both boxes)
- IIS7 on intranet box
- PHP 5.2.9 via FastCGI

Thanks in advance.
//CALL RECORDING PATH
$callRecordingSource = '\\\\192.168.1.15\recordings$';

//TRAVERSE DIRECTORY
foreach(new RecursiveIteratorIterator(new RecursiveDirectoryIterator($callRecordingSource)) as $entry){
//COPY FROM NETWORK TO TEMPORARY LOCATION
copy($entry->getPathname(),sys_get_temp_dir() . 'crec_' . md5($entry->getFilename()) . '.tmp');
}

Open in new window

Avatar of gr8gonzo
gr8gonzo
Flag of United States of America image

Hmmm, how frequent are the timeouts? Can you easily reproduce it?

If so, you should try creating a batch file that copies 20 or 30 files in a row over the network share just to see if the same problem happens outside of PHP.
On this page, http://us.php.net/manual/en/class.recursivedirectoryiterator.php , the last example shows breaking the process into two parts which is what I would be inclined to do.  I've had to do that a number of times to get PHP code to work right.  I also would make a test version that I can access thru a web page that tells me what it is doing with 'echo' statements.  Also, how busy is the computer that the web server is on?
Why not use an xcopy bat script instead? If these are intranet servers, I don't see the need for using php, especially since it's not designed for substantial execution times. The bat script can run for as long as it takes to complete the transfer without fear of timing out.
Avatar of psykomotor
psykomotor

ASKER

Thanks very much for the input:

- The timeout is in 90%+ of cases, the fact it works at all in a few rare cases is the strange part. (no code changes, just runs all of a sudden).

- The RecursiveDirectoryIterator runs fine, 100% of the time. It is only when the copy() function is called that this problem starts. The script echo's output through the whole process, and both servers are being worked on while no-one else is using them.

- I have been thinking about using a batch file after spending a whole working day yesterday trying to make this work. I have been trying out quite a few things with exec() again to no avail (appreciate you weren't suggesting calling it from PHP).

The record has to be inserted into a database, by interpeting the caller id and date/time using some regex on the filename which is why this started as PHP  - This much works absolutely perfect.

If the file would only copy at anywhere near the speed the windows GUI does, the execution time shouldn't have been a major problem.

However there's no reason not to use a batch script to copy the files across and have php simply work on the temporary dir. I actually have almost zero experience of batch scripts but now is as good a time as any to learn !

Thanks to everyone - I will get back once I've given the batch scripts a go.
There's something much weirder going on here...

I've started to work on a batch file which will:

- Traverse the network share for .wav files.
- Copy the file if it hasn't been modified within the last 60 seconds. (This stops active recordings being copied over).

(If xcopy can fulfill the second requirement that would be useful info as that's the reason I'm not using it).

However, something has gone disasterously wrong, and I wonder if its related. The "Command Prompt" window has totally jammed - Tried clicking the X, closing cmd.exe via task manager, closing the process, closing the process tree, using taskill with the image name, taskkill with the PID, pskill with the pid, and process assassin.

I've attempted to restart several times to no avail. It's a remote box so a hard reboot isn't an option. If anyone has any advice on rebooting the process, that would be great.

I accidentally ran the batch file again (so now have two jammed cmd.exe's !), but what I did notice is that the lockup occurs when the script access the share. i.e. at the 'for /r' line... I should point out this script ran perfectly around 10-15 times, then the freezing started all of a sudden.

I'm wondering if this the same lockup PHP was experiencing which caused it to time out. This is my first ever batch file, so it could just be bad code.

Any thoughts are much appreciated because I'm baffled !
::CLEAR SCREEN (DEBUGGING)
cls

::SET SOURCE FOLDER
set sourceFolder=\\192.168.1.15\recordings$\

::SET DESTINATION FOLDER
set destinationFolder=C:\RecordingsTemp

::TRAVERSE SOURCE FOLDER
for /r %sourceFolder% %%f in (*.wav) do call :checkAndConvert %%f %destinationFolder%

::---------------------------------------------

::SUBROUTINE - CHECK AND CONVERT
:checkAndConvert

echo %1

::END SUBROUTINE
goto:eof

::---------------------------------------------

Open in new window

Well, I've made some progress back on the PHP front, though I don't really know why this has worked:

- I removed the "recordings$" share and replaced it with "Recordings" (exactly as the folder appears on the disk). Whilst the speed still varies, it does seem to at least always work now.

To get round the unreliable speed, the script copies to a temporary location then checks if it has enough execution time to import the recording into the main system (create DB entries etc). I appreciate using microtime() isn't an accurate execution time check because of other processes etc, but it does a good job of interpeting the maximum CPU time that could have been used, which is suitable in this instance.

Whats interesting is I also built a little speed calculator for the debugging output which isn't accurate to the byte but gives a very good aproximation of speed. For no reason whatsoever, the speed will either be almost nothing (~10kbps) or a solid 5.5MB/sec.

If anyone has any info on why the speed fluctuates like this that would be much appreciated. The network is totally idle today, so there's no user demand.

I guess I'm going to have to visit the client's office and perform a hard reboot next week to get rid of the hung cmd.exe windows - they're not doing any 'apparent' harm but still...
If the speed is ~10kbps - does it actually end up copying anything? It sounds like the speed calculator is calculating bytes / time, but if the script hangs for 10 minutes and then recovers and copies 20 megabytes in 1 second, then you might end up with unreliable stats. Your script might think it was copying 20 megabytes over the entire 10 minutes and 1 second instead of a timeout followed by a burst.

Replacing the share doesn't sound like it SHOULD work. Maybe PHP has some trouble communicating with hidden shares, but it's unlikely. I don't see why it would make a difference from a technical perspective.

I'm still curious if the batch file would experience the same timeouts that you had before. Really, it sounds like a network configuration issue more than anything (e.g. improperly-configured router or switch, or faulty network devices). A switch that had bad instructions on how to route a packet between two points could cause a problem like that, or maybe someone modifying MAC addresses... Those types of things could definitely result in occasional packet timeouts until a "Where are you?" type of broadcast went out. But this type of problem would result in poor local network performance in general, so if you're frequently communicating back and forth between the server and the computer running the script, then you should see the network performance problem happen outside of the PHP script.

Regarding the hung processes - I'm assuming you're using teminal services / RD to run sessions where you've run these cmd.exe windows. If so, try logging into the console mode of RD (basically the "REAL" desktop, not a virtual session), then use Terminal Services Manager to kill your terminal server sessions that have those hung cmd.exe processes. That might work without visiting the office.
Also, just in case you didn't see this yet, you can set the timeout of your PHP script with the set_time_limit PHP function. If you're running this as a scheduled job, just make sure you don't set the time limit so high that you end up with a race condition (the next scheduled process kicks off while the first is still running, and they end up doing some duplicate work).
Yes, even when slow, it can copy. The stats indeed are just bytes/time, so it's not a solid test of network transfer speed, but is suitable for the example - I've just worked a set out manually and its pretty much right.

I think I'm basically there now with the solution as a whole, I've created a batch file on the PBX server that encodes the wav files using libsndfile into a compressed format. As they are only telephone conversations, you'd almost never tell from listenening, but they are only 10% of the size.

Oddly enough I was using set_time_limit() to *lower* the execution time - As it's on IIS7 with FastCGI, the FastCGI timeout was kicking in first and I had no debugging output to begin with !

I'm about to test run the whole thing from start to finish so, fingers crossed.

Thanks for the info about killing the cmd.exe processes - I'll have another go at shutting them down.

It's quite possible there is a problem with the network infrastructure - we've steadily been taking over from another company and there have been some... questionable... decisions with network settings, which we've been fixing as we find. For instance, the perimeter firewall was switched off, to name one !
ASKER CERTIFIED SOLUTION
Avatar of ee_auto
ee_auto

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial