We help IT Professionals succeed at work.

Any method to delete a line of text from a file using CFile or CStdioFile?

Fej
Fej asked
on
My application creates a log, as a text file, to record various events.  I append new entries, just a line of text, to the end of the file.  I would like to keep the log from growing over a certain size.  I was hoping that there was a way to delete the first line of text from the file (which would be the oldest entry) when the log gets to a certain byte size.  Thereby 'bumping' the rest of the file contents 'up.'

The only thing I can come up with is to read the whole dang log into memory remove the entry, add the new one and write the whole log back to the file.  There has got to be a more efficient method to do this.

Thanks,

Fej
Comment
Watch Question

ekc

Commented:
Yes there is... To use some database engine instead. :)
I'm kidding, but realy, I don't belive you can do it just like that, as you say - efficiently. If you realy wish to stay with log-files like this, you will have to make up some nice work-around.
Pure text file access is very low level service, and even if you find some piece of software with effective way to delete the first line of the file, leaving everything else intact, it would certainly have all this ugly and ineffective work behind the curtains...
ekc

Commented:
If you decide to implement a class performing something like this, I recommend you to do it not on entry basis (every exceeding line), but maybe on the 10 or 100-entry basis. Keep in mind that finding and opening file is the most time consuming operation in the file access!
Good luck.
write your log records as structures of a fixed size

eg - some filter info (date/time/errorlevel) and text description which is say 2K in length total then when you open your file you know that each log message starts at 2K boundaries

then you create your file with say n (blank) log messages (typically 1000 - or whatever).

then when you access the file you move to the first empty message an modify the contents - you could use a value at the beginning to tell you where to start in the file.

when you have reached the nth message you start at the beginning again

eg pseudocode

open file

read first long --> (n)
seek beginning
write n+1

seek file position n*2k
write log message

close file

you'll probably want to control access to you file so only one app (logger) can open it for writing at any time - or you could pass all this onto a logger app (object) which would sort this for you

 
CERTIFIED EXPERT
Author of the Year 2009

Commented:
When yoiu first open the log file, check if it is > 1000 lines.  If so, rewrite it to make it 900 lines and continue.

Then every time you add a line, see if it is more than 1000 lines.  As soon as it is, delete the first 100 lines before adding the lines.

In otherwords, add lines singly and delete lines in n-line blocks.

Also, you do not need to read the entire file into memory and then write it back out.  Just read the first 100 lines, then continue reading, but now write lines 101, 102, etc to the start of the file (save and restore the file pointer between read and write operations of course).

Do you need to see implementation code?

-- Dan
Fej

Author

Commented:
Dan,

Yes I would like to see the code.  I can only seem to be able to append to the end of a file or write over portions and not delete blocks or single lines.  Your comment suggests that you are able to selectively delete n blocks of text lines.  Or am I reading it wrong?

Thank you,
Graham
CERTIFIED EXPERT
Author of the Year 2009
Commented:
>> Your comment suggests that you are able to selectively
delete n blocks of text lines.  Or am I reading it wrong?

I don't think I said that.  I'm pretty suer that I said...

>> When you first open the log file, check if it is > 1000 lines.  If so, rewrite it to make it 900 lines and continue....

>> In otherwords, add lines singly and delete lines in n-line blocks.


Nuthin to it...
=-=-=-=-=-=-=-=-=-=-=-=-==-

void CD16GenDlg::OnButton1()
{
     CStdioFile fIn("c:\\temp\\test.txt", CFile::modeRead | CFile::shareDenyNone );
     CStdioFile fOut("c:\\temp\\test.txt", CFile::modeWrite | CFile::shareDenyNone  );

    CString s;
    for (int j=0; j<100; j++ ) {
        fIn.ReadString( s );
    }
    fOut.SeekToBegin();
    while ( fIn.ReadString( s ) ) {
        s+= "\n";
        fOut.WriteString( s );
    }
     fIn.Close();
    fOut.SetLength( fOut.GetPosition() ); // truncate to this len
    fOut.Close();
}
=-=-=-=-=-=-
Rather than keeping track of the file pointer, I found it easier to just open a second CFile.

-- Dan
don't forget however dans idea will sometimes have a short history, eg after the rewrite, and not the last n lines
Fej

Author

Commented:
Dan,

I see what you mean.  So to generalize:

1)  Use two file pointers to shift the contents of the file around to the way I want it.

That seems pretty straight forward.  I never thought of that.  I have it correct right?

Thanks,

Graham

Commented:
Here's a completely different approach (I've struggled with this task over and over again and finally have a really nice solution stuck in my low-level libraries):

We now use a rolling log -- one log for each of the last 10 days:

Create a class that reads the status (CFile::GetStatus()) of your log file (MYLOG.LOG). If the log is not dated today then, starting at MYLOG.LOG9, rename to 10. Rename 8 to 9, 7 to 8, etc until MYLOG.LOG is renamed MYLOG.LOG1.

Now open MYLOG.LOG and write to it.

This way your logs never get too big. If someone says "I had a problem on Tuesday" you can get at Tuesday's log.


My low-level logging class is derived from a CStdioFile and defaults the filename to the application's name so all I have to do is:

CLog myLog;
myLog.WriteString("blah blah blah");

wherever I need something logged.

An alternate constructor let's me specify a filename:

CLog myLog("CommErrs");
myLog.WriteString("No Connection");

In the low level code, I override WriteString to prepend the time that the entry was written.

Good luck!



Fej

Author

Commented:
Dan,

I see what you mean.  So to generalize:

1)  Use two file pointers to shift the contents of the file around to the way I want it.

That seems pretty straight forward.  I never thought of that.  I have it correct right?

Thanks,

Graham
CERTIFIED EXPERT
Author of the Year 2009

Commented:
>>1)  Use two file pointers to shift the contents of the file around to the way I want it.

Yes, that is what I am saying.  My other point is that you don't want to rewrite the whole (possibly gigantic) file with each new line added, so delete largish chunks to avoid needing to do this.  

An alternative is to just let the log grow unbounded during any given session, and chop it down to size at the start of the next session.

-- Dan
Fej

Author

Commented:
Thanks everyone for the help.  It appears that any of the answers would work.  But Dan's was certainly the best.

In summary, the answer to my question is NO, but that behavior can be simulated by using two or more file pointers.

Thanks all,

Graham
CERTIFIED EXPERT
Author of the Year 2009

Commented:
Just for completeness...
Do a keywordsearch in MSDN for
   FSCTL_SET_ZERO_DATA

NTFS provides a mechanism for "sparse file" operation.  I once played around with this and found that when a file has been marked for "Sparse" operations, you can effectively delete the first part of it by using...

   DeviceIOIoctl( FSCTL_SET_ZERO_DATA,... )

It took some digging and experimentation, because this is an addball function, but I finally found that if you "zero out" a full allocation unit (cluster) worth of data, you can effectively delete that cluster from the start of a file.  Reading data from the "start of the file" will actually start at the beginning of the un-zeroed cluster and NT will actually deallocate that cluster.

-- Dan