read/write locking protection under win32

I have a multi-threaded application which does a lot of file accessing. I wish to protect these files with a locking class, so that anyone can read the file as long as noone is writing and only one thread can be writing the file at one time.

I will have a whole list of files so I will have to maintain a thread safe list of locked files and protect them using the reads/writers locking method.

Does anyone have an implementation of the readers/writers lock for multiple files.  A solution should use win32 syncronisation routines.
ajh020797Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

nietodCommented:
Do you need to protect access to the entire file, or to byte ranges in the file.  Protrcting access to the entire file is trivial, you can just open the file for writting with no share or open it for reading with share.  Byte ranges is a  lot harder.  
0
AlexVirochovskyCommented:
see http://www.borland.com/borlandcpp/news/report/CR9403becker.html about
semaphors in multithread appa.
0
NickRepinCommented:
Just use LockFile to protect the range of bytes you want to read/write, then read/write them, them UnlockFile.
I can convert it to a simple C++ class if you like.
0
Cloud Class® Course: Microsoft Office 2010

This course will introduce you to the interfaces and features of Microsoft Office 2010 Word, Excel, PowerPoint, Outlook, and Access. You will learn about the features that are shared between all products in the Office suite, as well as the new features that are product specific.

NickRepinCommented:
NickRepin changed the proposed answer to a comment
0
NickRepinCommented:
The best way is LockFileEx, but it's for NT only.
0
GlennDeanCommented:
The only problem with LockFile is that it doesn't wait for the file to be unlocked, so one always has to write code like
BOOL gotIt=LockFile(--);
while (!gotIt)
{
 //Try again
 ::Sleep(--);
 gotIt=LockFile(--);
}

To avoid such code AND you only want one thread accessing the file at any instant in time, just use a mutex to guard the file, as in
HANDLE hFileNMutex=::CreateMutex(NULL,FALSE,"Nth File Mutex");
The above mutex is in a signalled state so at this point any thread can get to it (probably best to create it in your app's InitInstance).  

Then, any thread trying to get access to the Nth file just goes
::WaitForSingleObject(hFileNMutex,INFINITE);

As soon as this thread is released from the wait then you can be assured the mutex is non-signalled and you have complete control of the file.  Don't forget to call ::ReleaseMutex(hFileNMutex) when you're done with the file turn the mutex into the signalled state.
  Glenn
0
GlennDeanCommented:
LockFileEx solves the LockFile problem I indicated above and certainly seems to be a better solution than using mutexes.
0
nietodCommented:
The problem with LockFile() etc is that it doesn't allow a one writer multiple readers design.  The design needs to allow any number of readers at the same time but only one writter and (no readers) at the same time.  
0
NickRepinCommented:
Sorry, LockFile is not for multithreading. Ignore my comment.
0
NickRepinCommented:
Here is my version. Allows multiple readers/single writer.

#include <windows.h>

#pragma pack(push,4)
class Lock
{
public:

   Lock();
   ~Lock();

   void LockWrite();
   void UnlockWrite();


   void LockRead();
   void UnlockRead();

private:

   HANDLE hWMutex;   // Owned by writting thread.
   HANDLE hREvent;   // Signaled if can read.
   HANDLE hWEvent;   // Signaled if read operation is finished.

   // ****MUST BE ALIGNED ON DWORD****
   LONG   nRead;     // Number of reading threads.

};
#pragma pack(pop)

Lock::Lock()
{
   nRead=0;
   hWMutex=CreateMutex(0,FALSE,0);        // Not owned.
   hREvent=CreateEvent(0,TRUE,TRUE,0);    // Manual, signaled.
   hWEvent=CreateEvent(0,FALSE,FALSE,0);  // Auto, non-signaled.
}

Lock::~Lock()
{
   CloseHandle(hWMutex);
   CloseHandle(hREvent);
   CloseHandle(hWEvent);
}

void Lock::LockWrite()
{
   // At first, request mutex ownership.
   WaitForSingleObject(hWMutex,INFINITE);

   // Other writing threads cannot be here.

   // Restrict readings.
   ResetEvent(hREvent);

   // Now waiting for reading threads.

   // Can use this, but it takes significant CPU time.
   //while(nRead) Sleep(0);

   while(nRead) WaitForSingleObject(hWEvent,INFINITE);
}

void Lock::UnlockWrite()
{
   // Allow readings.
   SetEvent(hREvent);

   // Allow writings.
   ReleaseMutex(hWMutex);
}

void Lock::LockRead()
{
   for(;;) {
      nRead++;

      // Check if can read.
      if(WaitForSingleObject(hREvent,0)==WAIT_OBJECT_0) break;
      UnlockRead();
      WaitForSingleObject(hREvent,INFINITE);
   }
}

void Lock::UnlockRead()
{
   nRead--;
   SetEvent(hWEvent);
}


Lock lockForFile1;
Lock lockForFile2;
Lock lockForFileN;

void main(void)
{
   // To read from file X:
   lockForFileX.LockRead();
   ReadFile(...)
   lockForFileX.UnlockRead();
   
   // To write to file X:
   lockForFileX.LockWrite();
   WriteFile(...)
   lockForFileX.UnlockWrite();
}
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
NickRepinCommented:
Yet another version.

One note. nRead++ and nRead-- must be performed atomically.
It depends on compiler, and to be absolutely sure, you have to use InterlockedIncrement()/InterlockedDecrement() instead of nRead++ and nRead--;

#pragma pack(push,4)
class Lock
{
public:

   Lock();
   ~Lock();

   void LockWrite();
   void UnlockWrite();


   void LockRead();
   void UnlockRead();

private:

   CRITICAL_SECTION  cs;
   HANDLE hWEvent;   // Signaled if read operation is finished.

   // ****MUST BE ALIGNED ON DWORD****
   LONG   nRead;     // Number of reading threads.

};
#pragma pack(pop)

Lock::Lock()
{
   nRead=0;
   InitializeCriticalSection(&cs);
   hWEvent=CreateEvent(0,FALSE,FALSE,0);  // Auto, non-signaled.
}

Lock::~Lock()
{
   DeleteCriticalSection(&cs);
   CloseHandle(hWEvent);
}

void Lock::LockWrite()
{
   EnterCriticalSection(&cs);

   // Other writing threads cannot be here.

   // Can use this, but it takes significant CPU time.
   //while(nRead) Sleep(0);

   while(nRead) WaitForSingleObject(hWEvent,INFINITE);
}

void Lock::UnlockWrite()
{
   LeaveCriticalSection(&cs);
}

void Lock::LockRead()
{
   EnterCriticalSection(&cs);
   nRead++;
   LeaveCriticalSection(&cs);
}

void Lock::UnlockRead()
{
   nRead--;
   SetEvent(hWEvent);
}
0
nietodCommented:
That works for file locking, but not range locking.  That is why I was waiting to hear from ajh before trying to code anything  (range locking makes it quite a bit harder, I guess not that bad though.)

Nick, in your last version you are assuming that nRead-- is going to be atomic.  It might not be.

Usually this is done using a single mutex (in windows a critical section could be used) a read count and a write boolean.  The critical section protects access to the read count and the write boolean, but not to the file.  

If you need to range locking, then you will also need a table of lock ranges, also protected by the critical section.  This could be implimented as a map<> or sorted array that is sorted by the low value of the range and the stores with it the high value, or the length.  
0
NickRepinCommented:
>>That works for file locking, but not range locking

That's true. You are right - it's a little bit harder. Sure, you have to make a table of locks. I answered this Q because a) The range was not mentioned in the original question and b) I had a liitle bit of free time. ajh can reject my answer if he need to lock the range of bytes.
But I'm not sure that locking the range of bytes will increase the performance - it greatly depends on what ajh is going to do.
If ajh needs the range, you'll get a chance to answer this Q. I will not, because I have no so much time.

>>Nick, in your last version you are assuming that nRead-- is going to be atomic

I also assume that nRead++ in the 1st one is also atomic.
MS VC must use inc/dec instructions for this. In this case they will be atomic.

But I added the comments at the beginning of the 2nd version that nRead--++ must be atomic, it depends on compiler, so it's better to use InterlockedIncrement/Decrement.

<<Usually this is done using a single mutex (in windows a critical section could be used) a read count and a write boolean.  The critical section protects access to the read count and the write boolean, but not to the file.>>

Sorry, I did not fully understand this, especially the last sentence.
Critical section in my second answer makes sure that only one thread writes to the file.
nRead synchronizes the reading threads with the writing one.
0
nietodCommented:
>> can reject my answer if he need to lock the range of bytes.
Or give you the opportunity to modify the code first, which I always think is best.

>> I'm not sure that locking the range of bytes
>> will increase the performance
This might be a multithreaded application that needs to access the file from sepeate threads.  Range locking allows multiple threads to read from and write to the file "simulatnaeously" as long as a write location does not overlap any other location.  This may boost performance as threads don't have to wait for other threads to complete there read/write  (and this may lower peformance too, it depends on the level of activity.   This is definitely a good project to profile before and after the change!)

>> MS VC must use inc/dec instructions for this.
>> In this case they will be atomic.
I woudn't make that assumption.  First of all it s atomic on an x86 if it is performed in memory.  But the memory form takes 3 clocks.  What if the optimizer instead reads the value into a register, increments it and then writes it back out.  This is also 3 clocks, but it is possible that it may be used to allow more efficient use of the multiple instruction pipes.  (I don't remember the rules anymore, its been 5 years, but this might be possible.)

>> Critical section in my second answer
>> makes sure that only one thread
>> writes to the file. nRead synchronizes the
>> reading threads with the writing one.
I would use the critical section differently.  I woudl use it to protect both the read count and a boolean.  i.e you can't test or set either the read count unless you have locked the critical section.  The read count indicates the number of readers, as in yours, the boolean indicates if there is a writter.  (Alternately you could set the read count to -1 for this, I don't like that.)  So rather than keeping the critical section locked the entire time you are writting, you set the boolean to true.  Thus the critcal section is only locked when dealing with the read count and write boolean and is quickly unlocked again.
0
NickRepinCommented:
The problem with boolean is that reading threads and other writing ones must wait until write op is completed. We cannot use spin lock on boolean because of CPU overhead. With crit section, other threads will just wait.

Of course, may be, we need to perform other tasks instad of waiting. But again, it all greatly depends on what ajh is going to do.
0
NickRepinCommented:
Regarding inc/dec.
You are right, it will cause the problem on the multiple CPUs at least. InterlockedIncrement/Decrement is a must.

Regarding locking the range of bytes.
Sure, it may be useful if we read/write only one record at a time in a large file.
But suppose, that we need to overwrite entire file or change the contents of the file in several places. In this case the lock map may be useless.
0
nietodCommented:
>> The problem with boolean is that reading threads
>> and other writing ones must wait until write op is
>> completed. We cannot use spin lock on boolean
>> because of CPU overhead. With crit section, other
>> threads will just wait.
A good point, when I had to impliment this before that was not a concern, but it certainly should have been.

>> But suppose, that we need to overwrite entire file
>> or change the contents of the file in several places.
That is why I asked if it was needed...   Very useful if it is needed, very wasteful if it is not.
0
RadlerCommented:
Listening...
0
WynCommented:
Listening...
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Development

From novice to tech pro — start learning today.