Solved

File Locking in Java

Posted on 2011-03-09
8
333 Views
Last Modified: 2012-08-14
I have 2 threads polling the same directory for files to process - once a thread sees a file it will read the file name and start processing it - it keeps the file name in a cache so that it does not process it again. Once the middle tier has finished processing the file it signals the database to start processing it. This is through a status change of the file in the database itself - the database is polling this status and once it changes it starts processing.

The issue is there could be another thread polling the same directory - I need a way for a thread to lock the file once it has started processing it. How can I do this in Java in an O/S independent fashion ?

Thanks
0
Comment
Question by:abuyusuf35
8 Comments
 
LVL 26

Accepted Solution

by:
ksivananth earned 250 total points
ID: 35084258
0
 

Author Comment

by:abuyusuf35
ID: 35086971
Thanks for the response - my understanding is that the behaviour of this API is not consistent on all platforms - is that correct ?
0
 
LVL 26

Expert Comment

by:ksivananth
ID: 35087100
yes, but as long as all your processes aquires the lock, it should be fine in all platforms!
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 
LVL 10

Assisted Solution

by:gordon_vt02
gordon_vt02 earned 250 total points
ID: 35109520
You could also have one thread responsible for polling for new files.  Whenever it finds a file it can add it to a work queue that your other threads pull from.  If you use one of the BlockingQueue implementations in java.util.concurrent, you can use the put() and take() methods to control the synchronization or the timed offer() and poll() methods if you want threads to only wait for a certain amount of time.  This way you can avoid file locking entirely and be guaranteed consistent across platforms.
public class FilePoller {
    // number of files the queue can hold
    private static final int FILE_QUEUE_CAPACITY = 100;
    // number of processing threads to create
    private static final int PROCESSING_THREADS = 5;
    
    // executor for the polling thread
    private ExecutorService pollExecutor;
    // executor for the processing threads
    private ExeuctorService processingExecutor;
    // the polling thread
    private Poller poller;
    // the processing threads
    private List<Processor> processors;
    
    private final AtomicBoolean stopRequested = new AtomicBoolean(false);
    
    public void pollFiles() {
        pollExecutor = Executors.newSingleThreadExecutor();
        processingExecutor = Executors.newFixedThreadPool(PROCESSING_THREADS);
        
        // create a blocking queue to hold the files, producers will block until
        // queue is not full, consumers will block until queue is not empty
        BlockingQueue<File> fileQueue = new ArrayBlockingQueue<File>(FILE_QUEUE_CAPACITY);
        
        poller = new Poller(fileQueue);
        pollExecutor.submit(poller);
        processors = new ArrayList<Processor>();
        for (int i=0; i < PROCESSING_THREADS; i++) {
            Processor p = new Processor(fileQueue);
            processingExecutor.submit(p);
            processors.add(p);
        }
        
        try {
            // wait 5 minutes for both pollExecutor and processingExecutor to complete (max wait 10 minutes)
            pollExecutor.awaitTermination(5, TimeUnit.MINUTES);
            processingExecutor.awaitTermination(5, TimeUnit.MINUTES);
        } catch (InterruptedException ie) {
            // handle interrupt
        }
    }
    
    public void stop() {
        if (stopRequested.compareAndSet(false, true)) {
            pollExecutor.shutdown();
            processingExecutor.shutdown();
            poller.requestStop();
            for (Processor p : processors)
                p.requestStop();
        }
    }
    
    private static class Poller implements Runnable {
        private final BlockingQueue<File> queue;
        private final AtomicBoolean stopped = new AtomicBoolean(false);
        
        public Poller(BlockingQueue<File> queue) {
            this.queue = queue;
        }
        
        public void requestStop() {
            stopped.set(true);
        }
        
        public void run() {
            while (!stopped.get()) {
                try {
                    // poll for files....
                    // when file found:
                    queue.put(file);
                } catch (InterruptedException ie) {
                    // handle interrupt
                }
            }
        }
    }
    
    private static class Processor implements Runnable {
        private final BlockingQueue<File> queue;
        private final AtomicBoolean stopped = new AtomicBoolean(false);
        
        public Processor(BlockingQueue<File> queue) {
            this.queue = queue;
        }
        
        public void requestStop() {
            stopped.set(true);
        }
        
        public void run() {
            while (!stopped.get()) {
                try {
                    File f = queue.take();
                    processFile(f);
                } catch (InterruptedException ie) {
                    // handle interrupt
                }
            }
        }
        
        public void processFile(File file) {
            // process file
        }
    }
}

Open in new window

0
 
LVL 10

Expert Comment

by:gordon_vt02
ID: 35689697
I recommend splitting points between ksivanath's comment (35084258) and my comment (35109520) as both represent valid solutions to the problem.
0
 
LVL 53

Expert Comment

by:Dhaest
ID: 35759368
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
Does your audience prefer people in photos or no people? How can you best highlight what you’re selling? What are your competitors doing, and what can you do that is different and unique from them?  Continue reading to learn how to make your images …
The viewer will learn how to dynamically set the form action using jQuery.
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question