I'm going to try and put this into a story at the end, but here's the technical rundown of what I'm needing to do. Currently we have 3 different scanner platforms and 3 different 'Export' apps written in Delphi to post data into a DB for other workflow applications to process. My current task is to basically take the 3 'exporters' and combine them all into a single all-encompassing application.
First thing each of these exporters currently do is check for active 'job's' in the ExportJob table (each Exporter has it's own database currently which will be changed to ONE DB handling all 3). It then goes out to the ExportDirectory for each 'Job' which contains the files output by the individual scanners for that job. It then takes those files, opens them, grabs data from them and then stores them in the DB which allows the 'Import' app for that particular job to grab that data and import it into the workflow.
Here's my problem. I can get this to work with ONE DB, but one of the scanner platforms which scans a majority of the work will cause the other 2 to have to wait before processing their work. Currently all 3 process at the same time (3 different applications running at once). I suggested to management this problem and that we should have 3 instances of this new rewrite going at the same time (1 instance for each scanner type). I was told to make this work with ONE instance of the application and to use Threading to handle the work.
I'm not sure what 'best practice' would be on this, so I'm tapping into the best minds in the business for suggestions/comments, ect...
Here's my 'real-life' story to hopefully make the processing steps make more sense:
There is a street which has many Wholesale Supply warehouses (Scanner Platforms). Each warehouse has very loyal customers (ExportJobs) that show up many times per day to pick up supplies (data files ready for exporting) these loyal customers NEVER do business with the other warehouses. Throughout the day each warehouse stages the supplies for each loyal customer (scanner output files). The customer comes by and knows right where to do to get what's been staged for him (ExportDirectory). He takes the supplies home and waits for all the other loyal customers in the world to finish picking up their supplies (each customer is very nice and fair and only starts working when everyone else is ready). At that time each customer starts creating the end-product for his customers (workflow apps).
(Step 1 of process): All customers get their supplies before any customer begins making end products..
I need to make sure that Loyal Customer 'A' can walk into warehouse 'A' and handle business at the same type that Loyal Customer 'B' works with warehouse 'B' and handles business. This needs to work for any number of warehouse/customers being able to do business at the same time (C, D, E, ect...).
(Step 2 of process): All customers have their supplies and now it's time to start manufacturing products..
All the different loyal customers can begin assembling their supplies and shipping out the result to THEIR customers (storing data in main DB for workflow apps to pick up) at the same time and not have to wait until the 'Warehouse A' customers (of which there are TONS of) are done before the 'Warehouse B' customers (just a handful) can begin their work.
I hope I didn't confuse the heck out of everybody here. What I'm looking for is what the best practice for accomplishing something like this is, whether that would be multi-threading or some other method. I also want to know if this is accomplished through multi-threading how do I handle the termination of the threads and make sure they ARE terminated OR can I create a dynamic # of threads at the begining and reuse them over and over (just kill them off when the app is stopped and closed).
Thank you to ANYONE who looks at this and takes the time to assist and comment. I appreciate it.