?
Solved

How to create a segmented Textfile?

Posted on 2003-03-18
12
Medium Priority
?
327 Views
Last Modified: 2010-04-16
Can someone exlain a good and super fast method for writing Text into the middle of a Textfile?
0
Comment
Question by:Okey
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +2
12 Comments
 
LVL 7

Expert Comment

by:billious
ID: 8159638
I severely doubt this can be done, if I read it aright.

I'd blockread the original up to the insertion point, writing out to a second "modified" file, then blockwrite the insertion, then blockread the remainder of the original and blockwrite it to modified.

delete the original and rename the modified and the job's done.

If you want to update in-place, then don't waste your time - you can't just insert bytes into the middle of a file.

...Bill
0
 
LVL 6

Accepted Solution

by:
MannSoft earned 100 total points
ID: 8160919
I think it could also be done without rewriting the file.  Something like:

-Seek to insertion point
-Blockread all the text up to the end of the file
-Seek to insertion point + sizeof(insertion text)
-Blockwrite the text you just blockread
-Seek to insertion point
-Blockwrite insertion text

This way you move the end of the file down X bytes making room for the new text.  On a large file where you cant make a single blockread you would have to work it a little differently.  You'd have to move the end of the file down block by block, starting with the very end and working up towards the insertion point.  That would ensure you dont overwrite any text that hasnt been moved yet.
0
 
LVL 15

Expert Comment

by:VGR
ID: 8161320
agree with the above.
personally, for "super-fast" method I would (stupidely) use this :
-Assign / $I- / ReSet / $I+ / IOResult=0
-ReSet
-blockread first part,
-rewrite(newfile)
-blockwrite first part
-blockwrite inserted part
-blockread second part
-blockwrite second part
-close both
-delet first file
-rename newfile
0
On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

 
LVL 22

Expert Comment

by:grg99
ID: 8162036
If the new text is the exact same length as the old text, then it's easy.

Otherwise it's kinda slow.

A better question would be:  why do you want to do this, and would you consider alternatives.

0
 
LVL 7

Expert Comment

by:billious
ID: 8162385
Mannsoft:

Good idea, with a couple of drawbacks:

to use the method you suggest, you must be able to blockread a sufficient amount to memory. Suppose the insert-into position is @1K, length 1K in a 300M  file, for instance. Has the machine got 300M of available memory? If you write out the extra data to another file (even via paging) then you defeat the purpose.

The second problem is power or system failure, which Mr. Murphy dictates will happen any time after commencing to blockwrite. The file is then corrupted, since the block from insertionpoint + insertionsize will have been partially-written.

...Bill
0
 
LVL 6

Expert Comment

by:MannSoft
ID: 8162523
In the last paragraph I already gave the alternative method for working with large files.  And having to insert a 1k chunk at the 1k mark of a 300m file, both ways will have pretty close to the same amount of reads/writes.  Inserting a 1k chunk at the 299.9m mark of a 300m file one way will outperform the other by leaps and bounds.

But you're right about power failure.  If you want speed I think my way is going to be the faster of the two (although Im not quite interested enough to actually benchmark them <g>).  If you want the extra safety then rewriting the file each time is definitely the way to go.  Worse case scenario there is power failure after the delete, but you'll still have the rewritten file you can manually rename after rebooting.
0
 
LVL 7

Expert Comment

by:billious
ID: 8162716
Mannsoft:

[academic]

I suppose I stopped reading as soon as I found what I regarded as a problem with the method. It would certainly be faster than re-writing the whole file (and use less space, since only originalsize + insertionsize would be required, against (2*originalsize) + insertionsize for the reconstruct method.

Nevertheless, I believe we are agreed that the extra safety offered by the rewrite method indicates that is the better path.

As ever, it's a question of swings and roundabouts.

...Bill
0
 
LVL 1

Author Comment

by:Okey
ID: 8163216
OK I see I must explain the problem better!

I used to transfer a TEXT to a FILE VAR
(
 Copying Handle and RecSize into a FILE VAR
plus setting MODE to 55219
in something like this:

TFILEREC=RECORD
 HANDLE ,
 MODE,
 RECSIZE,
 BUFPOS{in TEXT ONLY},
 BUFEND{in TEXT ONLY}:WORD
 Garbage...
END;

Search the Help of Borlands TP7.0
for TTextRec and TFileRec
as FileRec or so ?????

)
then I use Seek, FILESIZE, FILEPOS, Blockread & Blockwrite with the new Filehandle and when I return to TEXT-VAR then I write only BufPOS:=BUFEND & READLN 2 times a Line
so TEXT-var would be positioned and so on!

Now my problem is that this is somehow buggy!
Fileposition of Lines in Text-VAR are somehow movin bytewise away when repeated seeked and READOUT with
ReadLn
How can I verryfied seek and search with this FastForward method into a textfile to especially copy segments from the center of several MB great TEXT-FILES
Segments have to be recognized, so if I seek too far I seek back until I am at least two readln's before the segment!
Segment is for analysis mentioned text that contains information that should be explained into it but only just segmentwise per loop!
I know Blockread is suiteable for Fast File IO
But It has to run with Textfiles too!
And  at Least I surely use IORESULT and so on!
0
 
LVL 15

Expert Comment

by:VGR
ID: 8164808
so don't use a textfile but a file of byte ;-)
0
 
LVL 1

Author Comment

by:Okey
ID: 8165312
I have to use TEXT-File,
because I Implemented several thousands of lines with!
Now I'm searching for a fast forward seek method to find the Segment in TEXT-File for copy, analyse and explain purposes. It's a source Text and has to be stuffed on with informations especially in the center!
I changed as far as possible to untyped FILE-var,
but I won't rebuild the complete TEXT-File-Handling
with EOF FLUSH WRITELN READLN (FILE- OPEN, WRITE/READ, FLUSH AND CLOSE)
Because it would lead to several hundreds of source lines!
I mean Borlands TP7,0 always does a Fileread when You use a Read/Ln no matter of the buffer
(
 Buffer will always be reloaded
)
That's why Text is too slow for a search through.
because the SetTEXTBuiffer sets only the maximum possible READ-Record for a File but doesn't do the read from buffer!

So if someone knows here a good/cheap trick
I'll listen to!
0
 
LVL 1

Author Comment

by:Okey
ID: 8165389
Think about this:
You want to stuff a sorted File
with numered lines.
The Numbered lines have to be continously and in order written.
To add new lines in between the old I'll have to seek a verrified position I don't know,
so that I've to do a fast scan forward in TEXT-File.
but while it's growing up to several MB, I thought about a stepped seek forward and scanin on as far as the destination is behind the read position
elsewise I'm seeking back to last acceptable position before!
The Bug is that the Destination position is movin away while doing this.
I mean the Suggested line stops several bytes away from with Filepos found position.
I'am using the following :

TYPE{Definition of MZ-EXE header}
 TEXT_PTR=^TEXT;

  PTextBuf = ^TTextBuf;
  TTextBuf = array[0..127] of Char;

  PTRTFileRec=^TFileRec;
  TFileRec = record
    Handle  :                Word;
    Mode    :                Word;
    RecSize :                Word;
    Privat  :array[1..26] of Byte;
    UserData:array[1..16] of Byte;
    Name    :array[0..79] of Char;
  end;

  PTRTTextRec=^TTextRec;
  TTextRec = record
    Handle   :                Word;
    Mode     :                Word;
    BufSize  :                Word;
    Privat   :                Word;
    BufPos   :                Word;
    BufEnd   :                Word;
    BufPtr   :            PTextBuf;
    OpenFunc :             Pointer;
    InOutFunc:             Pointer;
    FlushFunc:             Pointer;
    CloseFunc:             Pointer;
    UserData :array[1..16] of Byte;
    Name     :array[0..79] of Char;
    Buffer   :            TTextBuf;
  end;

PROCEDURE SET_FILE_REC;
BEGIN
 WITH TYPED_FILE_RECORD Do
 Begin
  HANDLE:=TXTFILE^.HANDLE;
  MODE:=55219;{File mode for Reading/writing}
  RECSIZE:=1;
  FOR I:=0 TO 79 DO NAME[i]:=TXTFILE^.NAME[i];
 End;
END;

Procedure TEXTSEEK(TXTFILE:PTRTTextRec;POSITION:LONGINT);
VAR
 I:BYTE;
 TYPED_FILE_RECORD:TFILEREC;
 _FILE:^FILE;
Begin
 TXTFILE^.BUFEND:=0;
 With TXTFILE^ Do BufPos:=BufEnd;
 SET_FILE_REC;
 _FILE:=ADDR(TYPED_FILE_RECORD);
 REPEAT SEEK(_File^,POSITION); UNTIL IORESULT=0;
END;

Function TEXTPOS(TXTFILE:PTRTTextRec;RECORD_SIZE:WORD):LONGINT;
VAR
 I:BYTE;
 TYPED_FILE_RECORD:TFILEREC;
 _FILE:^FILE;
Begin
 SET_FILE_REC;
 _FILE:=ADDR(TYPED_FILE_RECORD);
 WITH TXTFILE^
  Do REPEAT
      TEXTPOS:=FILEPOS(_File^)-BUFEND+BUFPOS;
     UNTIL IORESULT=0;
END;

Function TEXTSize(TXTFILE:PTRTTextRec;RECORD_SIZE:WORD):LONGINT;
VAR
 I:BYTE;
 TYPED_FILE_RECORD:TFILEREC;
 _FILE:^FILE;
Begin
 SET_FILE_REC;
 _FILE:=ADDR(TYPED_FILE_RECORD);
 REPEAT TEXTSize:=FILESIZE(_File^);UNTIL IORESULT=0;
END;

Did I Implemented here a Bug ?
0
 
LVL 1

Author Comment

by:Okey
ID: 8165411

oops I mean aehm this lines
(I'm sorry for old garbage I placed in the comment above!)

Procedure TEXTSEEK(TXTFILE:PTRTTextRec;POSITION:LONGINT);
VAR
 I:BYTE;
 TYPED_FILE_RECORD:TFILEREC;
 _FILE:^FILE;
Begin
 TXTFILE^.BUFEND:=0;
 With TXTFILE^ Do BufPos:=BufEnd;
 SET_FILE_REC;
 _FILE:=ADDR(TYPED_FILE_RECORD);
 REPEAT SEEK(_File^,POSITION); UNTIL IORESULT=0;
END;

Function TEXTPOS(TXTFILE:PTRTTextRec;RECORD_SIZE:WORD):LONGINT;
VAR
 I:BYTE;
 TYPED_FILE_RECORD:TFILEREC;
 _FILE:^FILE;
Begin
 WITH TYPED_FILE_RECORD Do
 Begin
  HANDLE:=TXTFILE^.HANDLE;
  MODE:=55219;{File mode for Reading and Reading/writing}
  RECSIZE:=RECORD_Size;
  FOR I:=0 TO 79 DO NAME[i]:=TXTFILE^.NAME[i];
 End;
 _FILE:=ADDR(TYPED_FILE_RECORD);
 WITH TXTFILE^
  Do REPEAT
      TEXTPOS:=FILEPOS(_File^)-BUFEND+BUFPOS;
     UNTIL IORESULT=0;
END;

Function TEXTSize(TXTFILE:PTRTTextRec;RECORD_SIZE:WORD):LONGINT;
VAR
 I:BYTE;
 TYPED_FILE_RECORD:TFILEREC;
 _FILE:^FILE;
Begin
 WITH TYPED_FILE_RECORD Do
 Begin
  HANDLE:=TXTFILE^.HANDLE;
  MODE:=55219;{File mode for Reading and Reading/writing}
  RECSIZE:=RECORD_Size;
  FOR I:=0 TO 79 DO NAME[i]:=TXTFILE^.NAME[i];
 End;
 _FILE:=ADDR(TYPED_FILE_RECORD);
 REPEAT TEXTSize:=FILESIZE(_File^);UNTIL IORESULT=0;
END;
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An introduction to the wonderful sport of Scam Baiting.  Learn how to help fight scammers by beating them at their own game. This great pass time helps the world, while providing an endless source of entertainment. Enjoy!
In this blog post, we’ll look at how using thread_statistics can cause high memory usage.
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…
In this video, Percona Director of Solution Engineering Jon Tobin discusses the function and features of Percona Server for MongoDB. How Percona can help Percona can help you determine if Percona Server for MongoDB is the right solution for …
Suggested Courses

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question