Borgs8472
asked on
Basic service to process CSV files and run bash scripts
So I have a series of bash scripts that take various parameters.
An external system drops parametrised CSV to a pick up location.
I need a basic service that I will pick up these simple CSV files when then arrive and pass them over to the scripts and execute them.
What's a light weight way to execute this? I'd rather not have a ultra frequently cron job, rather something reactive to the arrival of the file.
An external system drops parametrised CSV to a pick up location.
I need a basic service that I will pick up these simple CSV files when then arrive and pass them over to the scripts and execute them.
What's a light weight way to execute this? I'd rather not have a ultra frequently cron job, rather something reactive to the arrival of the file.
ASKER
I'm new to Linux (windows background) but I always found services were more robust than scheduled tasks.
Also, I don't want more than one instance of the job running at any one time (jobs should be processed sequentially, not concurrently) and I worry running at a cron could allow jobs to run out of control which mustn't happen.
Also, I don't want more than one instance of the job running at any one time (jobs should be processed sequentially, not concurrently) and I worry running at a cron could allow jobs to run out of control which mustn't happen.
Cron is a service (crond). Depending upon what distribution you are using, you can start or stop it. In Red hat family of OSes, you can do
service crond start
service crond stop
service crond restart
Running multiple services at a time is not a problem but if you must not run them concurrently, you can do so by either checking if a particular process is running or keeping a lock file.
service crond start
service crond stop
service crond restart
Running multiple services at a time is not a problem but if you must not run them concurrently, you can do so by either checking if a particular process is running or keeping a lock file.
ASKER
Right... so I think I'm almost there, I can write a loop to search for the files, check lock conditions and spawn the scripts - so far so good.
But this script itself, I just need to ensure it's always running. Should I be terminating and re initiating it with cron? Isn't there a more graceful way to ensure my loop script is always running than that?
But this script itself, I just need to ensure it's always running. Should I be terminating and re initiating it with cron? Isn't there a more graceful way to ensure my loop script is always running than that?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks, looks like I've got a framework for my task queuing system now!
If you want to make your own service, that would be working in addition to cron, to basically do that same thing. The service already running in the system is cron, that does this kind of job. Any allergy to cron?