Solved

File Locking in Java

Posted on 2011-03-09
8
350 Views
Last Modified: 2012-08-14
I have 2 threads polling the same directory for files to process - once a thread sees a file it will read the file name and start processing it - it keeps the file name in a cache so that it does not process it again. Once the middle tier has finished processing the file it signals the database to start processing it. This is through a status change of the file in the database itself - the database is polling this status and once it changes it starts processing.

The issue is there could be another thread polling the same directory - I need a way for a thread to lock the file once it has started processing it. How can I do this in Java in an O/S independent fashion ?

Thanks
0
Comment
Question by:abuyusuf35
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 26

Accepted Solution

by:
ksivananth earned 250 total points
ID: 35084258
0
 

Author Comment

by:abuyusuf35
ID: 35086971
Thanks for the response - my understanding is that the behaviour of this API is not consistent on all platforms - is that correct ?
0
 
LVL 26

Expert Comment

by:ksivananth
ID: 35087100
yes, but as long as all your processes aquires the lock, it should be fine in all platforms!
0
WordPress Tutorial 1: Installation & Setup

WordPress is a very popular option for running your web site and can be used to get your content online quickly for the world to see. This guide will walk you through installing the WordPress server software and the initial setup process.

 
LVL 10

Assisted Solution

by:gordon_vt02
gordon_vt02 earned 250 total points
ID: 35109520
You could also have one thread responsible for polling for new files.  Whenever it finds a file it can add it to a work queue that your other threads pull from.  If you use one of the BlockingQueue implementations in java.util.concurrent, you can use the put() and take() methods to control the synchronization or the timed offer() and poll() methods if you want threads to only wait for a certain amount of time.  This way you can avoid file locking entirely and be guaranteed consistent across platforms.
public class FilePoller {
    // number of files the queue can hold
    private static final int FILE_QUEUE_CAPACITY = 100;
    // number of processing threads to create
    private static final int PROCESSING_THREADS = 5;
    
    // executor for the polling thread
    private ExecutorService pollExecutor;
    // executor for the processing threads
    private ExeuctorService processingExecutor;
    // the polling thread
    private Poller poller;
    // the processing threads
    private List<Processor> processors;
    
    private final AtomicBoolean stopRequested = new AtomicBoolean(false);
    
    public void pollFiles() {
        pollExecutor = Executors.newSingleThreadExecutor();
        processingExecutor = Executors.newFixedThreadPool(PROCESSING_THREADS);
        
        // create a blocking queue to hold the files, producers will block until
        // queue is not full, consumers will block until queue is not empty
        BlockingQueue<File> fileQueue = new ArrayBlockingQueue<File>(FILE_QUEUE_CAPACITY);
        
        poller = new Poller(fileQueue);
        pollExecutor.submit(poller);
        processors = new ArrayList<Processor>();
        for (int i=0; i < PROCESSING_THREADS; i++) {
            Processor p = new Processor(fileQueue);
            processingExecutor.submit(p);
            processors.add(p);
        }
        
        try {
            // wait 5 minutes for both pollExecutor and processingExecutor to complete (max wait 10 minutes)
            pollExecutor.awaitTermination(5, TimeUnit.MINUTES);
            processingExecutor.awaitTermination(5, TimeUnit.MINUTES);
        } catch (InterruptedException ie) {
            // handle interrupt
        }
    }
    
    public void stop() {
        if (stopRequested.compareAndSet(false, true)) {
            pollExecutor.shutdown();
            processingExecutor.shutdown();
            poller.requestStop();
            for (Processor p : processors)
                p.requestStop();
        }
    }
    
    private static class Poller implements Runnable {
        private final BlockingQueue<File> queue;
        private final AtomicBoolean stopped = new AtomicBoolean(false);
        
        public Poller(BlockingQueue<File> queue) {
            this.queue = queue;
        }
        
        public void requestStop() {
            stopped.set(true);
        }
        
        public void run() {
            while (!stopped.get()) {
                try {
                    // poll for files....
                    // when file found:
                    queue.put(file);
                } catch (InterruptedException ie) {
                    // handle interrupt
                }
            }
        }
    }
    
    private static class Processor implements Runnable {
        private final BlockingQueue<File> queue;
        private final AtomicBoolean stopped = new AtomicBoolean(false);
        
        public Processor(BlockingQueue<File> queue) {
            this.queue = queue;
        }
        
        public void requestStop() {
            stopped.set(true);
        }
        
        public void run() {
            while (!stopped.get()) {
                try {
                    File f = queue.take();
                    processFile(f);
                } catch (InterruptedException ie) {
                    // handle interrupt
                }
            }
        }
        
        public void processFile(File file) {
            // process file
        }
    }
}

Open in new window

0
 
LVL 10

Expert Comment

by:gordon_vt02
ID: 35689697
I recommend splitting points between ksivanath's comment (35084258) and my comment (35109520) as both represent valid solutions to the problem.
0
 
LVL 53

Expert Comment

by:Dhaest
ID: 35759368
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0

Featured Post

Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Australian government abolished Visa 457 earlier this April and this article describes how this decision might affect Australian IT scene and IT experts.
The SignAloud Glove is capable of translating American Sign Language signs into text and audio.
Progress
Starting up a Project
Suggested Courses

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question