mavmanau
asked on
creating a server or process that will automatically fire off tasks according to file uploaded
Hi,
Firstly thank you for taking a look at my question. Apologies for the lengthy title, it was hard to shorten down what I am looking to do.
Essentially I want to create a web service of some sort, that will allow various clients to upload csv files, that will then get auto-imported into a mysql DB, and would fire off messages to destinations listed in the uploaded csv, and will then check results and write them back to the sql server.
The main thing i guess that I want to work out, is how to create a worker process as such. I can easily write the code to recieve the csv file etc. and i have written the code that will be required to go out and send msg's, and check the results. It is the "worker" process that i do not know how to approach ie would it be a service that always runs?
just looking to see if anyone can point me in the right direction.
many thanks!
Firstly thank you for taking a look at my question. Apologies for the lengthy title, it was hard to shorten down what I am looking to do.
Essentially I want to create a web service of some sort, that will allow various clients to upload csv files, that will then get auto-imported into a mysql DB, and would fire off messages to destinations listed in the uploaded csv, and will then check results and write them back to the sql server.
The main thing i guess that I want to work out, is how to create a worker process as such. I can easily write the code to recieve the csv file etc. and i have written the code that will be required to go out and send msg's, and check the results. It is the "worker" process that i do not know how to approach ie would it be a service that always runs?
just looking to see if anyone can point me in the right direction.
many thanks!
I would use the curl_multi_.... functions to send off all the requests simultaneously (akin to what Ray was saying about running a script asynchronously). Here's a good article that covers it:
http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/
http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/
Hi There mavmanau, , , I read through your question and understand the "create a Service" part and the "add the uploaded csv file to my Database" part , but in the " fire off messages to many destinations in the csv file", I do not exactly follow what you need your service to do, especially for the "check results and write them back" part.
let's say you have this in your csv -
gonzo@hotmail.com,1002:0cb f:8a53:005 2:1c00:da2 e:307c:dd3 4,www.musicdata.com/rest,www.matter.com/api
with 4 web destinations, the first is an email, the second is an IP address for a SOAP WSDL service, the third is a web address for a REST result, the third is a web address for a JSON-RPC result
as you can see, I am not sure what you may be trying to do since all of these have a different result, and what kind of "check results" you may mean.
if you do not have many destinations, you can do them one at a time, but since you say "worker process", I guess that you want to handle these destinations as separate processes? You might set up different PHP pages to do each different "Type" of result, and use a cURL to those pages, for a muti-task sort of thing. But some more info about what the "fire off messages" might mean, could be helpful.
let's say you have this in your csv -
gonzo@hotmail.com,1002:0cb
with 4 web destinations, the first is an email, the second is an IP address for a SOAP WSDL service, the third is a web address for a REST result, the third is a web address for a JSON-RPC result
as you can see, I am not sure what you may be trying to do since all of these have a different result, and what kind of "check results" you may mean.
if you do not have many destinations, you can do them one at a time, but since you say "worker process", I guess that you want to handle these destinations as separate processes? You might set up different PHP pages to do each different "Type" of result, and use a cURL to those pages, for a muti-task sort of thing. But some more info about what the "fire off messages" might mean, could be helpful.
ASKER
Hi,
Sorry for the lengthy time away, i had to jet away for a little bit.
I have thought a bit more about what I am trying to achieve and it is more easily summarised like this:
I want to send SMS's via a gateway, but I want to pace them according to whichever data gets uploaded. so for example, one file could be uploaded, and they want the messages split up so that they deliver one msg every 7 seconds or so, or they might want a job to start at 12pm and it is only 10am and they upload the file.
I can get the data written into a SQL db quite easily, with the details like the start time and all of that, but I would need a webpage or service that then checks the DB to see if a msg is due to be sent, and then fire it off if needed. but I would also need the ability if I wanted to pause the paced job, then pick up again, once it starts.
so the only part would be how would i get that service/web page/worker process to do the periodical checks to see if msg's need to go out etc.
hope this makes sense.
many thanks
Sorry for the lengthy time away, i had to jet away for a little bit.
I have thought a bit more about what I am trying to achieve and it is more easily summarised like this:
I want to send SMS's via a gateway, but I want to pace them according to whichever data gets uploaded. so for example, one file could be uploaded, and they want the messages split up so that they deliver one msg every 7 seconds or so, or they might want a job to start at 12pm and it is only 10am and they upload the file.
I can get the data written into a SQL db quite easily, with the details like the start time and all of that, but I would need a webpage or service that then checks the DB to see if a msg is due to be sent, and then fire it off if needed. but I would also need the ability if I wanted to pause the paced job, then pick up again, once it starts.
so the only part would be how would i get that service/web page/worker process to do the periodical checks to see if msg's need to go out etc.
hope this makes sense.
many thanks
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
ASKER
Thank you. Your help was much appreciated.
1. Client request uploads a CSV
2. Server response
2.a. Saves CSV to DB
2.b. Sends requests with messages to destinations
2.c. Receives responses from destinations
2.d. Records responses to DB
2.e. Create the "Thank you" page for the upload.
An alternative might be to abstract steps 2.b through 2.d and put these into a separate script that can be started with a cURL POST request or fsockopen(). This script could run asynchronously. Once the uploaded CSV is saved, you could tell the client that the data is being processed, and the client would not have to wait for the remote destinations to complete their work.