Solved

PHP , FIFO queues and Collection objects

Posted on 2006-07-24
10
937 Views
Last Modified: 2012-08-13
Hi,

I am looking for a piece of php code that will take 3 incoming files namely src.rtf,trg.rtf and file.email. These files when they arrive via file upload will be given the same Timestamp as they belong to the same work order. They will then be placed into some sort of Collection and placed into a FIFO queue.

As objects are taken off the queue (first in - first out) it should then be possible to take the Object and retrieve the 3 files so that they can be processed.


Any help appreciated.

Kind regards,

Paul
0
Comment
Question by:plambkin1
  • 7
  • 3
10 Comments
 
LVL 40

Expert Comment

by:RQuadling
ID: 17166713
Are the files uploaded at the same time?

As the files have to be saved somewhere, I don't really see the need for a DB.

To handle the file uploading, I would create a folder with the datetime stamp from

microtime(True);

This folder would exist in a folder called Queue.

I would use the number as the value is not really important. You may want to convert the time to a human readable Y-m-d format, but not necessary.

Then using move_uploaded_file() to move the files (one at a time) from the upload temp directory to the new folder.

Pretty much done.

For the queue handler, I would get all the folder names in Queue (PHP5's DirectoryIterator class will do quite nicely) into an array.

Once all the names are retrieved, sort them in numeric order.

Then process them 1 at a time in sequence.

As you process one, either move the files to a folder called Completed/timestamp or delete them if they are no longer needed.


0
 
LVL 40

Expert Comment

by:RQuadling
ID: 17166714
Is that the sort of thing you are looking for? Any sort of changes to that?
0
 

Author Comment

by:plambkin1
ID: 17166839
Hi,

Thanks a million. Something like this. Any idea on what the code for PHP5's DirectoryIterator class would look like.
0
 
LVL 40

Expert Comment

by:RQuadling
ID: 17166859
<?php
$a_dirs = array();
foreach(new DirectoryIterator('./Queue') as $obj_FILE)
 {
 if (($obj_FILE !== '.') && ($obj_FILE !== '..') && $obj_FILE->isDir())
  {
  $a_dirs[] = $obj_FILE->getPathname();
  }
 }
sort($a_dirs);

// $a_dirs now contains all the folders in FIFO order.
?>

0
 

Author Comment

by:plambkin1
ID: 17167747
Thanks,

If microtime(true) is used and the files are placed into this folder then I presume that this php code will then exit

or is it the case that the DirectoryIterator will be rebuilt each time a new set of files is added?

Kind regards,

Paul




0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 40

Expert Comment

by:RQuadling
ID: 17167869
Yes it will exit.

The queue will be handled by another program.

The DI will rebuild each time, but each time it does, it will be a new set of folders as it will have processed the existings ones first.

It could simply loop and run forever. And then use a third program to monitor the state of the DI one.

0
 

Author Comment

by:plambkin1
ID: 17167878
I don't think this is going to work. I have a DOS process which detects whether there is a file in the <<incoming>> folder. If there is, it is processed and the file is deleted from the <<incoming>> queue.

I need something that will detect that the <<incoming>> queue is now empty and pop the first element of the FIFO and put it in there. I have a feeling that this is not going to work with PHP?

Kind regards,

Paul
0
 
LVL 40

Expert Comment

by:RQuadling
ID: 17168024
Use a BATCH file.

So you have folders like ...

QUEUE (populated by PHP as previously discussed)
WORK_DIR (the directory where your DOS app needs files to process).

And then batch file ...

@ECHO OFF
SET QUEUE_DIR=C:\Queue
SET WORKING_DIR=C:\Working

REM Is the working directory empty?
:WatchQueue
SET WORKING_CONTENT=##Empty##
FOR %%F IN (%WORKING_DIR%\*.*) DO SET WORKING_CONTENT=%%F
IF _%WORKING_CONTENT%_==_##Empty##_ THEN GOTO GetFIFO
GOTO WatchQueue

:GetFIFO
SET FIFO_CONTENT=##Empty##
REM Get the most oldest entry in the queue.
FOR /F %%F IN ('DIR %QUEUE_DIR%\*.* /O-D /-P') DO SET FIFO_CONTENT=%%F
REM If there is nothing in the queue at the moment, keep waiting until there is.
IF _%FIFO_CONTENT%_==_##Empty##_ GOTO GetFIFO

ECHO Going to move %FIFO_CONTENT%
MOVE /Y %QUEUE_DIR%\%FIFO_CONTENT% %WORKING_DIR%\%FIFO_CONTENT%
GOTO WatchQueue



Run this as a service and you have the problem solved (I think).

The only issue I would have is that saving the files in PHP takes a finite amount of time. You don't want to be in the process of doing this and have the batch file move the files.

So, in PHP, create the file with just the timestamp. WHen all the files are uploaded rename the directory to timestamp.FIFO

And in the batch file change

FOR /F %%F IN ('DIR %QUEUE_DIR%\*.* /O-D /-P') DO SET FIFO_CONTENT=%%F


to

FOR /F %%F IN ('DIR %QUEUE_DIR%\*.FIFO /O-D /-P') DO SET FIFO_CONTENT=%%F


So only .FIFO files are looked for.

A FIFO file will only be available when no actions are outstanding on it (all files will be closed).

MOVE ing and RENAME ing folders is (as far as I know) an atomic action. Nothing interrupts it.






0
 
LVL 40

Accepted Solution

by:
RQuadling earned 500 total points
ID: 17168071
Ah. Move the contents of the timestamp.FIFO folder, not the folder. And a better FOR command.

@ECHO OFF
SET QUEUE_DIR=C:\Queue
SET WORKING_DIR=C:\Working

REM Is the working directory empty?
:WatchQueue
SET WORKING_CONTENT=##Empty##
FOR %%F IN (%WORKING_DIR%\*.*) DO SET WORKING_CONTENT=%%F
IF _%WORKING_CONTENT%_==_##Empty##_ THEN GOTO GetFIFO
GOTO WatchQueue

:GetFIFO
SET FIFO_CONTENT=##Empty##
REM Get the most oldest entry in the queue.
FOR /F %%F IN ('DIR %QUEUE_DIR%\*.* /B /O-D /-P') DO SET FIFO_CONTENT=%%F
REM If there is nothing in the queue at the moment, keep waiting until there is.
IF _%FIFO_CONTENT%_==_##Empty##_ GOTO GetFIFO

ECHO Going to move %FIFO_CONTENT%
MOVE /Y %QUEUE_DIR%\%FIFO_CONTENT%\*.* %WORKING_DIR%

REM Clean up
RD %QUEUE_DIR%\%FIFO_CONTENT%

REM Loop
GOTO WatchQueue



The FOR command does a DIR on the folder and shows only the filenames in reverse date order - the oldest at the bottom - and this is the last one to update the env variable which is used in the next line.

This requires XP or 2000 (maybe 98 don't know) to work.
0
 
LVL 40

Expert Comment

by:RQuadling
ID: 17168098
OOI. Why not just put all the files in the directory with unique names?

Ah! The app wants specific filenames and doesn't iterate.

Most likely.

NOTE: The batch file I've given could easily be a PHP script. Both of which could be installed as services within a windows server or workstation. Running permanently.

0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Generating table dynamically is the most common issue faced by php developers.... So it seems there is a need of an article that explains the basic concept of generating tables dynamically. It just requires a basic knowledge of html and little maths…
Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now