Read/Write/Update a file at same time

I am allowing users on my website to place their vote.  Their submissions go to my VOTE.CGI perl script.  It opens the VOTESUB.DAT file to add/update their vote tally.

Now if I have millions of users accessing VOTE.CGI and thus VOTESUB.DAT all at the same time, wouldn't VOTESUB.DAT get all screwed up?  I've got to hold back the user submissions and handle them one at a time so as to not screw up the numbers held with in it.  Is this true?  

If so I have thought about writing out a dummy file called HOLD.TMP everytime VOTE.CGI is started.  So, everytime VOTE.CGI is started it checks for the existence of HOLD.TMP.  If it exists then we know that another occurence of VOTE.CGI is still running right now.  I must 'pause'(sleep) until this file has dissapeared (deleted by that previous instance of VOTE.CGI once it completes itself).  When the file is gone immediately create a HOLD.TMP file (so as to pause any next VOTE.CGI instances) and now do the handling of the voter submission.

Is this the best method to handle multiple writes to a single file?  Or is there another method to pause all clients on a web page so as to do stuff like this?

One more thing: is it true that the only way to update a file (ie my VOTESUB.DAT) is to read it all in (or in parts) and then write it all out, even if all you wanted to do is add a single value at the half way point in the file?  So if So lets say VOTESUB.DAT looked like:
1: Hello Line 1
2: Hello Line 2
3: Hello Line 3

If I wanted to alter line 2 to read: Hello From Line 2
I MUST read in ALL of VOTESUB.DAT and write it back out again?!


Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

The first answer is: file locking- probably using flock(). (This was a FAQ here a couple of months ago).

Your hold.tmp idea is most people's first thought, but it won't work since two processes could write the file simultaneously. Much better to use actual file locking (though the standard "flock" method won't work on every system.

On the second question, you don't need to read and rewrite the entire file if the records are fixed length and you don't need to insert (just update). If you need inserts and have a lot of data, a dbm file (which is effectively a hash tied to a file) is a good way to do it. But i am assuming in your application your data file will never get much bigger than a few Kbytes, so I wouldn't bother with anything fancy since you wouldn't be saving much time and would be making the whole thing more complicated. So unless you use a DBM file the answer is yes, you have to read the entire thing in.

Here's all you need to do to update the file: (assumes the vote to be added is in $thisvote)

$LOCK_EX = 2; # Exclusive
open(DAT, "+<votesub.dat") || die "Can't open file: $!";
flock(DAT,$LOCK_EX) || die "Can't flock: $!";
# put a sleep(5); here to test locking, run script twice at same time
while (<DAT>) {
seek(DAT, 0, 0)  || die "Can't seek: $!";
foreach (keys %votearray) {
 print DAT "$_=$votearray{$_}\n";
 print "$_=$votearray{$_}\n";
close(DAT)   || die "Can't close: $!";

That's all you need... you do need to place a sleep there to verify flock works on your system - call the script multiple times and verify each waits for the one before it to finish and the data is correct, the 5 second sleep makes it easy to tell.

Good luck!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
oops, left in a debugging statement that echos the results to the screen:
print "$_=$votearray{$_}\n"

I tested it by adding $thisvote=$ARGV[0]; at the beginning and just calling it from the command line, this is a simple way to see how it works.

mirrorAuthor Commented:
Once again, alamo.  Amazing.  I am off to try out what youi said.  Looks like I've got to change lots of code.

CompTIA Cloud+

The CompTIA Cloud+ Basic training course will teach you about cloud concepts and models, data storage, networking, and network infrastructure.

mirrorAuthor Commented:
Alamo, I have many more questions pertaining to DBM and FLOCK.  I can't seem to find decent info on them (everything book references another reference).  Please, can you email me directly so that I can talk to you.  I need answers quickly.
mirrorAuthor Commented:
Alamo, what if I use my old method of locking my VOTESUB.DAT file but I handle itt this way.

When its ok, I write out the HOLD.TMP file as usual but inside this file I write the $$ (process Id #).  Then after writing this file, I read it in.  Is it my process Id #?  If not, continue waiting....If so continue on and handle the VOTESUB.DAT file.

Does this resolve the problem of writing the HOLD.TMP file at the same time?  

Merry Christmas.
Your altered method is better but still not foolproof, since you can't completely ensure that someone won't come along and overwrite it after you verify the value has been written to it. Also, it's unclear whether what you read back is your own buffer or not.

flock is explained in the core perl documentation, it's pretty straughtforward. Are you saying the code above doesn't work on your system?
mirrorAuthor Commented:
Here's my actual code I'm using Alamo.  Just before I actually use this CGI program, I call with SETLIGHT(DBASE,RED).  It checks if all is well then moves on in.  Is this not foolproof?  As you look at this I'll run off and check the FLOCK command in the perl docs.  Thanks for all your help....  And sorry for the messy code below.  It's probably better viewed if you copy the text and paste it in a text editor.  SOme of those comments seem to run pretty long.

#Check to see if we can handle data now
# All is ok now, handle program here

sub SetLight()

  if ($who eq "DBASE")   #We need to set a light on FTP.DAT

  if ($type eq "RED")  #Set an indicator to tell other FTP2.CGI                         that we're using FTP.DAT
  { my($tm)=1;
    while (-e "tmp/$fname")               #If we're not the first instance of FTP2.CGI, we must wait until the previous is done
    {$tm=($tm==1)?2:1; sleep($tm);}
    open (OUT,">tmp/$fname");             #The marker is off, quickly jump in by creating marker and becoming TOP priority.
    print OUT "$$"; close(OUT);           #Notice we write the PROCESS ID #.  A unique # for this process.

    open(IN,"tmp/$fname");                #We now verify that we wrote the HOLD file.  This prevents writes at same time to this file.  Verify PROCESS ID #
    $num=<IN>; close(IN);
    if ($num!=$$) {goto AGN;}             #If process ID isn't ours return back to the loop and wait
  else                                    #Green light. Kill any markers to allow other FTP2.CGI in

mirrorAuthor Commented:
Any really quickly.  My database files which I created under DBM format (using the SDBM_FIle) are huge.  I am placing 5000 items in the one databse which has about 32 bytes per line.  I did some math and got around 160K.  But my DBM file is around 1.1 Megs.  Whats up with that and PLEASE assure me that this isn't all loaded into memory.  I really don't know much about servers but I'd hate for my website to be sucking all the servers memory and have me banned from my ISP.  (I left a more detailed message on the PERL section - HUGE DBM FILES)

Tanx again, man.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Scripting Languages

From novice to tech pro — start learning today.