Solved

PHP , FIFO queues and Collection objects

Posted on 2006-07-24
10
935 Views
Last Modified: 2012-08-13
Hi,

I am looking for a piece of php code that will take 3 incoming files namely src.rtf,trg.rtf and file.email. These files when they arrive via file upload will be given the same Timestamp as they belong to the same work order. They will then be placed into some sort of Collection and placed into a FIFO queue.

As objects are taken off the queue (first in - first out) it should then be possible to take the Object and retrieve the 3 files so that they can be processed.


Any help appreciated.

Kind regards,

Paul
0
Comment
Question by:plambkin1
  • 7
  • 3
10 Comments
 
LVL 40

Expert Comment

by:RQuadling
Comment Utility
Are the files uploaded at the same time?

As the files have to be saved somewhere, I don't really see the need for a DB.

To handle the file uploading, I would create a folder with the datetime stamp from

microtime(True);

This folder would exist in a folder called Queue.

I would use the number as the value is not really important. You may want to convert the time to a human readable Y-m-d format, but not necessary.

Then using move_uploaded_file() to move the files (one at a time) from the upload temp directory to the new folder.

Pretty much done.

For the queue handler, I would get all the folder names in Queue (PHP5's DirectoryIterator class will do quite nicely) into an array.

Once all the names are retrieved, sort them in numeric order.

Then process them 1 at a time in sequence.

As you process one, either move the files to a folder called Completed/timestamp or delete them if they are no longer needed.


0
 
LVL 40

Expert Comment

by:RQuadling
Comment Utility
Is that the sort of thing you are looking for? Any sort of changes to that?
0
 

Author Comment

by:plambkin1
Comment Utility
Hi,

Thanks a million. Something like this. Any idea on what the code for PHP5's DirectoryIterator class would look like.
0
 
LVL 40

Expert Comment

by:RQuadling
Comment Utility
<?php
$a_dirs = array();
foreach(new DirectoryIterator('./Queue') as $obj_FILE)
 {
 if (($obj_FILE !== '.') && ($obj_FILE !== '..') && $obj_FILE->isDir())
  {
  $a_dirs[] = $obj_FILE->getPathname();
  }
 }
sort($a_dirs);

// $a_dirs now contains all the folders in FIFO order.
?>

0
 

Author Comment

by:plambkin1
Comment Utility
Thanks,

If microtime(true) is used and the files are placed into this folder then I presume that this php code will then exit

or is it the case that the DirectoryIterator will be rebuilt each time a new set of files is added?

Kind regards,

Paul




0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 40

Expert Comment

by:RQuadling
Comment Utility
Yes it will exit.

The queue will be handled by another program.

The DI will rebuild each time, but each time it does, it will be a new set of folders as it will have processed the existings ones first.

It could simply loop and run forever. And then use a third program to monitor the state of the DI one.

0
 

Author Comment

by:plambkin1
Comment Utility
I don't think this is going to work. I have a DOS process which detects whether there is a file in the <<incoming>> folder. If there is, it is processed and the file is deleted from the <<incoming>> queue.

I need something that will detect that the <<incoming>> queue is now empty and pop the first element of the FIFO and put it in there. I have a feeling that this is not going to work with PHP?

Kind regards,

Paul
0
 
LVL 40

Expert Comment

by:RQuadling
Comment Utility
Use a BATCH file.

So you have folders like ...

QUEUE (populated by PHP as previously discussed)
WORK_DIR (the directory where your DOS app needs files to process).

And then batch file ...

@ECHO OFF
SET QUEUE_DIR=C:\Queue
SET WORKING_DIR=C:\Working

REM Is the working directory empty?
:WatchQueue
SET WORKING_CONTENT=##Empty##
FOR %%F IN (%WORKING_DIR%\*.*) DO SET WORKING_CONTENT=%%F
IF _%WORKING_CONTENT%_==_##Empty##_ THEN GOTO GetFIFO
GOTO WatchQueue

:GetFIFO
SET FIFO_CONTENT=##Empty##
REM Get the most oldest entry in the queue.
FOR /F %%F IN ('DIR %QUEUE_DIR%\*.* /O-D /-P') DO SET FIFO_CONTENT=%%F
REM If there is nothing in the queue at the moment, keep waiting until there is.
IF _%FIFO_CONTENT%_==_##Empty##_ GOTO GetFIFO

ECHO Going to move %FIFO_CONTENT%
MOVE /Y %QUEUE_DIR%\%FIFO_CONTENT% %WORKING_DIR%\%FIFO_CONTENT%
GOTO WatchQueue



Run this as a service and you have the problem solved (I think).

The only issue I would have is that saving the files in PHP takes a finite amount of time. You don't want to be in the process of doing this and have the batch file move the files.

So, in PHP, create the file with just the timestamp. WHen all the files are uploaded rename the directory to timestamp.FIFO

And in the batch file change

FOR /F %%F IN ('DIR %QUEUE_DIR%\*.* /O-D /-P') DO SET FIFO_CONTENT=%%F


to

FOR /F %%F IN ('DIR %QUEUE_DIR%\*.FIFO /O-D /-P') DO SET FIFO_CONTENT=%%F


So only .FIFO files are looked for.

A FIFO file will only be available when no actions are outstanding on it (all files will be closed).

MOVE ing and RENAME ing folders is (as far as I know) an atomic action. Nothing interrupts it.






0
 
LVL 40

Accepted Solution

by:
RQuadling earned 500 total points
Comment Utility
Ah. Move the contents of the timestamp.FIFO folder, not the folder. And a better FOR command.

@ECHO OFF
SET QUEUE_DIR=C:\Queue
SET WORKING_DIR=C:\Working

REM Is the working directory empty?
:WatchQueue
SET WORKING_CONTENT=##Empty##
FOR %%F IN (%WORKING_DIR%\*.*) DO SET WORKING_CONTENT=%%F
IF _%WORKING_CONTENT%_==_##Empty##_ THEN GOTO GetFIFO
GOTO WatchQueue

:GetFIFO
SET FIFO_CONTENT=##Empty##
REM Get the most oldest entry in the queue.
FOR /F %%F IN ('DIR %QUEUE_DIR%\*.* /B /O-D /-P') DO SET FIFO_CONTENT=%%F
REM If there is nothing in the queue at the moment, keep waiting until there is.
IF _%FIFO_CONTENT%_==_##Empty##_ GOTO GetFIFO

ECHO Going to move %FIFO_CONTENT%
MOVE /Y %QUEUE_DIR%\%FIFO_CONTENT%\*.* %WORKING_DIR%

REM Clean up
RD %QUEUE_DIR%\%FIFO_CONTENT%

REM Loop
GOTO WatchQueue



The FOR command does a DIR on the folder and shows only the filenames in reverse date order - the oldest at the bottom - and this is the last one to update the env variable which is used in the next line.

This requires XP or 2000 (maybe 98 don't know) to work.
0
 
LVL 40

Expert Comment

by:RQuadling
Comment Utility
OOI. Why not just put all the files in the directory with unique names?

Ah! The app wants specific filenames and doesn't iterate.

Most likely.

NOTE: The batch file I've given could easily be a PHP script. Both of which could be installed as services within a windows server or workstation. Running permanently.

0

Featured Post

Easy Project Management (No User Manual Required)

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
Part of the Global Positioning System A geocode (https://developers.google.com/maps/documentation/geocoding/) is the major subset of a GPS coordinate (http://en.wikipedia.org/wiki/Global_Positioning_System), the other parts being the altitude and t…
The viewer will learn how to count occurrences of each item in an array.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now