Solved

Controlling access to a file

Posted on 2001-06-08
25
201 Views
Last Modified: 2010-05-02
Hi,

I have a relatively complex situation here. What I want to do is to have a file on the hard drive. This file can be read and written to by an arbitrary number of programs on the computer. What I would like to do is maintain a system of read and write locks on the file. The way I envision it is that any program that requests read access to the file will be given access as long as no one currently has a write lock or has a pending write lock. When someone requests a write lock, no more read locks will be granted and once all existing read locks have been released the write lock will be granted. If possible I would like to suspend the execution of requestors until their lock request is granted. How can I do this?

Note that I have taken a quick look at the mutex which works well for single entry as well as the waitforsingleobject which suspend execution until the state changes. Both of these may come in handy.

Also note that I do not want to have a server managing this. It all must be done by the requesting clients, preferably with the OS doing as much of the work as possible.

I realize this is a hard question and I am willing to up the points if necessary.

Zaphod.
0
Comment
Question by:Z_Beeblebrox
  • 12
  • 8
  • 2
  • +3
25 Comments
 
LVL 1

Expert Comment

by:ramy050799
ID: 6170293
Hi Zaphod,
It is a hard situation really but there's i simple idea i don't know if it will work with you or not.
When a client needs to write to the file you may take temp copy from the file with any other name (don't make this name fixed or you will face the same problem again) and then open it to the client for editing and when he's done copy it again to the original file.

hope this will work with you
Good Luck:))
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6170297
Hi ramy,

That would work fairly well except for a few problems:

- If two programs try to edit the file at the same time, one of them will overwrite the other's changes
- If the new file is copied over the original while someone is reading it, what will happen?
- Programs may read an out of date version of the file (this might not be a bad thing)

Zaphod.
0
 
LVL 2

Expert Comment

by:i_jester69
ID: 6170365
Write another file in the same directory, and maintain entries that describe what access is being granted, etc. at any given time.
Each client could check the entries, and act accordingly.

0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6170387
Hi,

Is there any way to queue up requests in windows, so each program that wants to access that file makes a request and then suspends until it is granted?

Zaphod.
0
 
LVL 3

Expert Comment

by:Hornet241
ID: 6170465
This isn't very elegant but it should work.

First create a Read/Write Lock file for the apps to register their requests ( A DB table would be much easier to allow for easy checks for who's turn)

When an app makes a request enable a timer set to what ever time you want and a Public variable that records it's place in line.

Chks the file/db for a write request and if there is deny a read.

Enter the apps request in the list and assign it the next number

On the Timer event check if the Number assigned is next in the list and then allow that app to have their request

When there turn comes up allow the app to do the read or the edit and when the app is finished delete the apps entry in the Read/Write list

0
 
LVL 2

Expert Comment

by:kmv
ID: 6170485
If you want to proxide such service to any program (not only to program written by you) you need to write system driver for HDD. But I think it is not good idea to do such service.
Read please MSDN article about file opening (system function CreateFile) http://msdn.microsoft.com/library/psdk/winbase/filesio_7wmd.htm
Program logic may be different. If developer of the program created program correctly, your service is needless. In some cases you will improve work of programs but in other you can corrupt its work. I think that good idea is to provide such service only to those programs which know about it.
E.g. you can create system service program and other applications can tell your service that they want to use your arbitrage operations for certain file(s). I think it is more correct than your first variant.
0
 
LVL 14

Expert Comment

by:Matti
ID: 6171674
Hi!

Basicly it is this same in here now, the Access driver has solved this like this:

http://support.microsoft.com/support/kb/articles/Q186/3/04.ASP

Now this is for database, not for regular files, and to make this work on every file need's similar prog like a virus protector is. These subclass system messages very wide, and VB is not very best choice to make such wide message hook.

Also if the "prosessor" is only in client macine, it can't detect these messages in the servers OS, so this is impossible.

Working Message hook dll's for VB are in Spy Work's pack.

So if this file is Access database this LDB reader will help other vice not much.

 
Matti
0
 
LVL 1

Expert Comment

by:ramy050799
ID: 6172743
Hi zaphod,
In my point of view some of these few problems are not problems at all and some you would have faced them too if all clients were editing the same file at the same time.

For example the second problem is not a problem at all because no body will be reading the original file at any time, every client needs to edit will have his own temp copy that will edit it so the original file won't be open at any time.

The third problem i don't know why the programs will read out of date file, it is always updated once the client complete editing, think what if all clients where editing the same file? the file won't be updated too while a client is editing it will be updated after the client finishes completely updating and the same here.

The first problem i guess could be solved too but let us know first the type of file you are talking about.


Ramy...
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6172752
Hi,

Basically what I am doing is making my own DB engine. As a result, I need to allow concurrent read access to the file, but only sequential write access during which no one can read. Right now I have come up with a messy but functional solution to the problem but I wonder if anyone can see any problems with it or any improvements.

I have created a second file which will be used to control access to the file. This file will store 5 variables:
a boolean to say if the file is locked (ie being edited or processes waiting to edit it)
a long containing the thread ID of the thread editing the file if any
three string contains comma delimited lists of the thread IDs of the threads reading the file, waiting to read the file and waiting to write the file.

I will also have a mutex which will control access to the control file. Any time a thread wants to read the file, they first must get the mutex for the control file. Then, they check to see if the file is locked. If it isn't, they add their thread ID to the list of threads reading the file then release the mutex and being reading the file. If it is, then they add their thread ID to the list of pending threads and suspend themself.

If a thread wants to edit the file, then they first get the mutex for the control file, then check if there are any threads reading the file. If there are none, then it sets its thread ID to be the one editing the file and sets the boolean to true and begins editing the file. If there are then it sets the boolean to true and appends itself to the list of threads pending to write then suspends itself.

Any time a process completes its operation (either read or write), it is responsible for resuming the next thread(s) by examining the various variables.

The only problem with this that I see is that it will fail catastrophically if one of the threads fail. The only way I see around this would be instead of suspending threads, have them go to sleep for a certain period of time then resume and verify that the threads with access to the file are still operating, but I am not sure how to do this.

So, any comments?

Zaphod.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6176729
I think you'd better create some logical layers for your application. One layer only read/write to file at certain position, other calculates position in a file using record index, combine records in a buffer to produce other operations, third - change data in a buffer according SQL clause, forth - parse clause text etc.
But, as far as I understand, you want to create simple database application. Main idea is that you work with one object to read/write file. So you don't need to open/close file in each thread. Thread allocates buffer for object to work and calls its method to read/write data from file position N. All arbitrage you will do inside object. Using object you can create cache of file in a memory or use different methods to access file. If you test enouph your object code and remove bugs - you will know that inside your object methods programs won't fall. You can lock only those parts of file which you are writing. You can create some oprimizations using this object etc.
If you are interested in this variant we can discuss some points more detaily.

Is it your school project?

Good luck!
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6176802
This isn't a school project, just something I want to do for "fun" and experience :) I am also working on a few other apps which require very basic database backends and I don't like using Access.

The problem I am facing is managing the access that different programs have to the database, to ensure that they don't conflict. The method I proposed above should work, but I discovered that I cannot seem to suspend the current process. Looking closer at the MSDN docs seems to indicate that the suspendthread api call only works on windows CE, which is odd. Does anyone know how to suspend the current process and then resume it on 95/98/ME/NT/2000?

Zaphod.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6177395
Process and thread are different things (as it is written in MSDN). You cannot stop other process but can stop/continue one of your process threads. You only can send messages to other applications. But I don't know system messages like 'Suspend' or 'continue'. I think that there is no such messages or they are undocumented.

As a matter of fact, database should solve all conflicts itself, e.g. Access (Jet) does it. As I wrote before, it is not good idea to manage other applications.
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6178474
Hi,

I do not want to suspend other processes, my clients (which I am writing) should suspend themselves. They can then be resumed by another process. Is this possible?

Alternatively, is there a way of determining the current usage of a semaphore? All I want to do is to suspend a process until the semaphore is no longer in use.

Zaphod.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6178674
ok. Let's talk about your task.
Do you customer want that you create manager not for all tasks but for some specific tasks which will be written by you or somebody other but they will be a part of some system? I mean situation when you create technology which you (or somebody other) will use for some customer(s).
1. If yes - it is more simple than you told before. I have wrote you about service which can be used to arbitrage programs to access to a certain system resource.
2. If no - you can suspend your own processes but (AFAIK) you cannot resume other process. Manager can only set/clear flags. Processes can see whether flags are set and continue to work or no.
For first variant there are a lot of methods to implement it. Please answer what of variants do you have.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6179031
Hi,

I am creating this system for my own use. What I want to created is a distributed system for controlling access to the file, so no central server. I am willing to use APIs but I don't want to have to run a service. I am creating a DLL which programs can reference which will manage all of the DB functions. Therefore I can trust the clients implicitly since I have a layer written by me between the user code and the file.

The programs that use the DLL may or may not be written by me.

The whole goal behind this exercise is to create a very easy to use DB, with no installation required, no services to run, etc. All that is needed is to register a DLL. I am somewhat disappointed that it is proving to be so difficult to do.

BTW, if I can suspend my own process, but cannot resume other processes, how can my process ever get resumed?

Zaphod.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6181289
I see that I told uncorrectly phrase.
You cannot suspend process. You can suspend one of threads of your process. You cannot suspend main thread of your process.
You can create named event, mutex or semaphore to allow or restrict access to resource for different processes. Then you will use system function WaitForSingleObject (or WaitForMultiplyObjects) to wait until resource will be released. (Don't forget to handle other events, e.g. DoEvents)
If you create COM objects you also can do it using COM events but it is slightly more difficult than using named syncronization objects.
MSDN:
CreateEvent http://msdn.microsoft.com/library/psdk/winbase/synchro_8ub8.htm
Syncronization objects http://msdn.microsoft.com/library/psdk/winbase/synchro_5gqb.htm
Wait functions
http://msdn.microsoft.com/library/psdk/winbase/synchro_5vqr.htm

Best regards.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6181323
Hi,

I have done some research into all of these. I can do everything with them except differentiate between exclusive access and non-exclusive access. What I mean by this is that if two programs are currently reading the file, and then a third wants to write to it, that program needs to wait for the first two to finish. So what should he wait on?

Zaphod.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6181490
You can use flag FILE_SHARE_READ in function CreateFile to restrict processes which wants to write to file.
Other variant (I think it is better for your case) is to use functions LockFile, LockFileEx and UnlockFile to lock part of file.
LockFile http://msdn.microsoft.com/library/psdk/winbase/filesio_63xh.htm
LockFileEx http://msdn.microsoft.com/library/psdk/winbase/filesio_39h4.htm
UnlockFile http://msdn.microsoft.com/library/psdk/winbase/filesio_1jl1.htm

Best regards.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6181554
I looked into those as well, but how do I wait until the lock is released. AFAIK those functions will just returns errors if the file is already locked, or in use and so the process must keep retrying. I would like to queue up requests rather than spin locking.

Zaphod.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6181951
To queue requests you have to create one object which will handle all your requests. Variants which I know how to do this: create service or COM server. COM server can be self-registered, as for service, you need to write some information to registry.
Your processes will send request messages to service (e.g. 'read x bytes from file y start from position z', 'write ...'). This queries can be placed to queue(s) (e.g. one file - one queue) and processed by service. To connect with clients you can use named pipes or COM interfaces. So you can send result of reading to clients from server or data to write from client to server.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6182294
But this requires running a service. Just to make sure there is no confusion, this will all be on one computer. I do not intend to support networked access to this file. As a result, things like mutexes and such work well for queuing and controlling access, I just don't know how to switch between concurrent access and exclusive access. Maybe there just is no way to do this using only windows without spinlocking, in which case I guess I will have to spinlock.

Zaphod.
0
 
LVL 2

Accepted Solution

by:
kmv earned 200 total points
ID: 6182468
Mutexes and events will spinlock processes. You wrote that you want to create queue of requests. To create correct 'queue' I see only one variant - to use one object which will get all requests and pass tham to execute to other object or do it itself. So I proposed you service or COM server.
But there are some other ways. If you want to use LockFile you will check whether part of file you need is locked or not and wait until it will released. It is a spinlock.
I'd like to use COM server in your case. But of course you have to decide yourself what way to use.
About network access - I didn't think that you want to support network access. I thought that you want to do it on local computer.
If you didn't read Jeffrey Richter's book 'Advanced Windows' please try to find it. I've got it 2 weeks ago and it helps me a lot.

Best regards.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6182789
Using a mutex will suspend the process, so busy waiting is not required. Mutexes also support FIFO queues of processes waiting to gain ownership. This is the sort of set up I want. But it is looking like I will have to go for a compromise. Here is what I am currently thinking.

Each process that goes to read the file creates a unique mutex and places its name in the control file. When it is done reading it releases the mutex and removes its name from the control file. When a process comes to write, it tries to get each of the existing mutexes sequentially.

Zaphod.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6182799
The benefit of the method I just described is that it will handle process that do not cleanly exit, since those mutexes will be freed by windows.
0
 
LVL 7

Author Comment

by:Z_Beeblebrox
ID: 6211741
Thanks for all your help. I tried to implement the method I described but it turned out to be a nightmare. As a result, for now I will only allow exclusive access to the file using a mutex. If it becomes a performance issue, I will revisit this.

Zaphod.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

You can of course define an array to hold data that is of a particular type like an array of Strings to hold customer names or an array of Doubles to hold customer sales, but what do you do if you want to coordinate that data? This article describes…
If you need to start windows update installation remotely or as a scheduled task you will find this very helpful.
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now