Link to home
Create AccountLog in
Avatar of mavmanau
mavmanauFlag for Australia

asked on

creating a server or process that will automatically fire off tasks according to file uploaded

Hi,

Firstly thank you for taking a look at my question.  Apologies for the lengthy title, it was hard to shorten down what I am looking to do.

Essentially I want to create a web service of some sort, that will allow various clients to upload csv files, that will then get auto-imported into a mysql DB, and would fire off messages to destinations listed in the uploaded csv, and will then check results and write them back to the sql server.

The main thing i guess that I want to work out, is how to create a worker process as such.  I can easily write the code to recieve the csv file etc. and i have written the code that will be required to go out and send msg's, and check the results. It is the "worker" process that i do not know how to approach ie would it be a service that always runs?

just looking to see if anyone can point me in the right direction.

many thanks!
Avatar of Ray Paseur
Ray Paseur
Flag of United States of America image

I think the "worker process" might be built into the rest of the scripts; it would not need to be a long-running job.  To paraphrase your description, here is how I am seeing the steps.

1. Client request uploads a CSV
2. Server response
2.a. Saves CSV to DB
2.b. Sends requests with messages to destinations
2.c. Receives responses from destinations
2.d. Records responses to DB
2.e. Create the "Thank you" page for the upload.

An alternative might be to abstract steps 2.b through 2.d and put these into a separate script that can be started with a cURL POST request or fsockopen(). This script could run asynchronously.  Once the uploaded CSV is saved, you could tell the client that the data is being processed, and the client would not have to wait for the remote destinations to complete their work.
Avatar of gr8gonzo
I would use the curl_multi_.... functions to send off all the requests simultaneously (akin to what Ray was saying about running a script asynchronously). Here's a good article that covers it:

http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/
Hi There  mavmanau, , , I read through your question and understand the "create a Service" part and the "add  the uploaded csv file to my Database" part , but in the " fire off messages to many destinations  in the csv file", I do not exactly follow what you need your service to do, especially for the  "check results and write them back" part.
let's say you have this in your csv -
gonzo@hotmail.com,1002:0cbf:8a53:0052:1c00:da2e:307c:dd34,www.musicdata.com/rest,www.matter.com/api

with 4 web destinations, the first is an email, the second is an IP address for a SOAP WSDL service, the third is a web address for a REST result, the third is a web address for a JSON-RPC result

as you can see, I am not sure what you may be trying to do since all of these have a different result, and what kind of "check results" you may mean.

if you do not have many destinations, you can do them one at a time, but since you say "worker process", I guess that you want to handle these destinations as separate processes? You might set up different PHP pages to do each different "Type" of result, and use a cURL to those pages, for a muti-task sort of thing. But some more info about  what the "fire off messages" might mean, could be helpful.
Avatar of mavmanau

ASKER

Hi,

Sorry for the lengthy time away, i had to jet away for a little bit.

I have thought a bit more about what I am trying to achieve and it is more easily summarised like this:

I want to send SMS's via a gateway, but I want to pace them according to whichever data gets uploaded. so for example, one file could be uploaded, and they want the messages split up so that they deliver one msg every 7 seconds or so, or they might want a job to start at 12pm and it is only 10am and they upload the file.

I can get the data written into a SQL db quite easily, with the details like the start time and all of that, but I would need a webpage or service that then checks the DB to see if a msg is due to be sent, and then fire it off if needed.  but I would also need the ability if I wanted to pause the paced job, then pick up again, once it starts.

so the only part would be how would i get that service/web page/worker process to do the periodical checks to see if msg's need to go out etc.

hope this makes sense.

many thanks
ASKER CERTIFIED SOLUTION
Avatar of Member_2_248744
Member_2_248744
Flag of United States of America image

Link to home
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
See answer
Thank you.  Your help was much appreciated.