Solved

Handling text files with streams

Posted on 2000-03-23
25
487 Views
Last Modified: 2010-04-04
To access text-files, Delphi has standard Read, ReadLn and Write, WriteLn functions to perform low level I/O. I would like to know if it is somehow possible to perform these operations on text-files using streams. Imagine this:

var
  Int1, Int2: Integer;
  fs: TFileStream;
  tf: TextFile;

// way1
Read(tf, Int1);
// way2
fs.ReadBuffer(Int2, SizeOf(Integer));

Now you can probably tell me that this is impossible to have Int1=Int2. I know that. But is there any way to treat stream object as text file (TTextStream)?
I need this feature because Iam forced to handle very old text-files and cannot modify their format and still use capabilites of streams.

Thanks in advance. Ivo.
0
Comment
Question by:ivobauer
  • 7
  • 6
  • 5
  • +3
25 Comments
 
LVL 9

Expert Comment

by:ITugay
ID: 2648669

You may use this:
-----------

procedure StringToStream(STM : TStream; const S : string);
var LEN : integer;
begin
  LEN:=length(S);
  STM.Write(LEN,SizeOf(LEN));
  STM.Write(pointer(S)^,LEN);
end;

function StringFromStream(STM : TStream) : string;
var LEN : integer;
begin
  STM.Read(LEN,SizeOf(LEN));
  SetLength(result,LEN);
  STM.Read(pointer(result)^,LEN);
end;
----

Also you can use TStringList.LoadFromFile and SaveToFile. It allow you to work with text files without streaming.

Cheers,
Igor.
0
 
LVL 2

Expert Comment

by:PeterLarsen
ID: 2648674
I dont understand what the problem is. Please give some more details.
0
 
LVL 2

Author Comment

by:ivobauer
ID: 2648748
I need to be able to read (extract values), modify and write text-files containing strings, dates, times, integers and reals. File snippet below:

//begin of file snippet
06-08-199809:15:31 1  400  80000  2000 499.8779  132400  400000 K 50O
// end of file snippet

As I wrote earlier, I can use standard Delphi text files I/O routines such as Read or Write. But I am wondering about if is it possible to read/write these text files with streams. Reading/writing of strings, dates, times to stream is no problem (I use routines similar to ITugay)but my problem is that I am unable to read for example integer or float value from a text file by stream - cannot determine where it starts or ends (Read function handles this automatically). All these files have the same format which I cannot change. I want to write class and implement methods such as:
TMyVeryOldFileFormat.LoadFromFile(FName: String);
TMyVeryOldFileFormat.LoadFromStream(Stream: TStream);
TMyVeryOldFileFormat.SaveToFile(FName: String);
TMyVeryOldFileFormat.SaveToStream(Stream: TStream);

Is it enough?
0
 
LVL 10

Expert Comment

by:Lischke
ID: 2648815
Hi all,

I would do it so:

type
  TMVOFFStream = class(TFileStream)
    function GetNext: String;
  end;

function TMVOFFStream.GetNext: String;

var
  C: Char;

begin
  Result := '';
  while Position < Size do
  begin
    Read(Char, 1);
    if not (Char in ['A'..'Z', 'a'..'z', '0'..'9', '.', ':', '-']) then Break;
    Result := Result + C;
  end;
end;

Ciao, Mike
0
 
LVL 9

Expert Comment

by:ITugay
ID: 2648822
It look like text database. Is it strongly formatted? I mean every column have fixed length or have constant separator?

Igor.
0
 
LVL 10

Expert Comment

by:Lischke
ID: 2648824
If performance is an issue then you need additional approaches like buffering a part or the entire content of the file.

Ciao, Mike
0
 
LVL 10

Expert Comment

by:Lischke
ID: 2648860
OOps, just found two typos. The code must be:

function TMVOFFStream.GetNext: String;

var
  C: Char;

begin
  Result := '';
  while Position < Size do
  begin
    Read(C, 1);
    if not (C in ['A'..'Z', 'a'..'z', '0'..'9', '.', ':', '-']) then Break;
    Result := Result + C;
  end;
end;

Ciao, Mike
0
 
LVL 2

Expert Comment

by:PeterLarsen
ID: 2648916
I agree with Igor. It's very important to know if it is a text file or a binary file - please answer Igor's questions.
If it's a text file, i would do something similar to the sample in Mikes comment.
You could simplify the load by using TStringList.LoadFromFile and then run through the text property.
If you want to save the files (TMyVeryOldFileFormat.SaveToFile(FName: String); + TMyVeryOldFileFormat.SaveToStream(Stream: TStream);) i would consider using a different format, e.g. Packed Record.

Regards
Peter
0
 
LVL 10

Expert Comment

by:Lischke
ID: 2648976
Very first words in a question to be answered:

    To access text-files, ...

--- end of quote

Ciao, Mike
0
 
LVL 2

Expert Comment

by:PeterLarsen
ID: 2649005
:-)
But the question dont say anything about whether it's a text-database or not - or does it ?!! :´)
0
 
LVL 9

Expert Comment

by:ITugay
ID: 2649015
Regarding my previous comment, I'm trying to understand one thing. Is it possible to work with that text file as with text database table or not. I assumed to place this idea in the following comment. But still have no answer;-)

Best regards,
Igor.
0
 
LVL 10

Expert Comment

by:Lischke
ID: 2649054
Ah yes :-) but a "text-database" is not a binary file. It is still text. Anyway, converting the files with Excel into dBase or Paradox format is a good idea but requires an extra step. Can TTable or TQuery directly handle such a "text-database"?

Ciao, Mike
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 2

Expert Comment

by:PeterLarsen
ID: 2649073
Yes, It is possible to handle asc-files (text files) with TTable.
0
 
LVL 9

Expert Comment

by:ITugay
ID: 2649108
hi Mike,
I never doubted that we talk about text file;-). I only want to know: "Is it formatted?". TTable CAN handle such file

//-------
06-08-1998 09:15:31 1  400  80000  2000 499.8779  132400  400000 K 50O
06-08-1998 09:15:31 1  400  80000  2000 499.8779  132400  400000 K 50O
06-08-1998 09:15:31 1  400  80000  2000 499.8779  132400  400000 K 50O
//------

without any converting, but then you need to "provide the field descriptor information by calling the function DbiSetFieldMap to set a field map or you can bind external schema to text tables...."

Cheers,
Igor.

0
 
LVL 2

Author Comment

by:ivobauer
ID: 2649136
Some explanation: It is a text-file with some pseudo-binary portions. It is impossible to treat it as database because it consists of variable-length columns. Structure of file is following:

1)File header - plain text
It starts with date and time values (stored as sequence of characters), and then some integers and floats (also encoded as plain text). Order of those values is predefined but amount of bytes they occupy can vary (according to their values).

2) Pseudo-binary part.
Let me explain it. It is sequnce of known number of chars stored as plaintext (output from A/D converter) which I transform to integers by simple transformation. Number of appearance of this parts in a file is determined.

Summary:

to read a number of chars from stream i use function:

function ReadCharsFromStream(Stream: TStream; Num: Integer): String;
var
  i: Integer;
  ch: char;
begin
  result:= '';
  if Num > 0 then
    for i:= 1 to Num do
    begin
      Stream.ReadBuffer(ch, sizeof(char));
      Result:= Result + ch;
    end;
end;

by using this function I can extract date and time character sequences and simply convert them to tdatetime by encodedate and encodetime. Similarly I can extract sequences of characters. But I don't know how to extract integer and float values which were stored in file as sequence of characters.

Is it enough?
0
 
LVL 2

Author Comment

by:ivobauer
ID: 2649152
Little note: this file format comes from MS QuickBasic era - ten years ago and I cannot modify it.
Ivo.
0
 
LVL 10

Expert Comment

by:Lischke
ID: 2649175
:-) thank you, I never worked with this kind of database, only larger ones like Sybase or Interbase. What I assume though is, that the steps involved to read the files via TTable are quite expensive. But I might be wrong so I will wait for a response from Ivo to know what effort he wants to make...

Ciao, Mike
0
 
LVL 10

Accepted Solution

by:
Lischke earned 50 total points
ID: 2649196
Oops, crossposting...


Ivo, with the routine I gave you, you can easily read out every part of the line. Since you know what format each value has you could do like:

var
  S: String;
  I: Integer;
  F: Single;
  D: TDateTime;

begin
  with Stream do
  begin
    // 06-08-1998 09:15:31
    S := GetNext;
    ConvertToDateTime(S);
    // 1  
    S := GetNext;
    I := StrToInt(S);
    // 400  
    S := GetNext;
    I := StrToInt(S);
    // 80000  
    S := GetNext;
    I := StrToInt(S);
    // 2000
    S := GetNext;
    I := StrToInt(S);
    // 499.8779  
    S := GetNext;
    F := StrToFLoat(S);
    // 132400  
    S := GetNext;
    I := StrToInt(S);
    // 400000
    S := GetNext;
    I := StrToInt(S);
    // K
    S := GetNext;
    // 50O
    S := GetNext;
    I := StrToInt(S);
    ... etc.
  end;
end;

Is that what you need?

Ciao, Mike
0
 
LVL 9

Expert Comment

by:ITugay
ID: 2649214
Ivo,
 it's only my opinion.
Is it possible to show small sample of your file (header and body). I will try to handle it as a table. The deal is that variable-length columns is not limitation to use text file as table. You only need to have constant delimiter.

Best regards,
Igor.

PS: certainly if all this is interesting to you.
0
 
LVL 2

Author Comment

by:ivobauer
ID: 2649254
Mike: Your last routine seems to suit my needs so I will try it. Thanks.

Igor: Thanks for your effort, but I don't need to treat this file as database table. But I can send you a sample. Wait, please.


0
 
LVL 2

Author Comment

by:ivobauer
ID: 2649279
=====Begin of sample=====
A205-04-199408:05:08 1  250  25000  1000  0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 6 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANebyl 1000  1000  10  0  0  0  0  3618  8.833008  31800  100000 E  1.+),-(*.)',,'),'',+',-),.+(.-)*,*(-)'*-'',+'*,(*-,',-)).,(-.().*',,'),(',,')-)'-,'-.*).,'+,)(-''*,'',*(+,'(-+',,)).+(,.').*',,'),''+,'),)(,,',.*'-,(*,)(-)')-(',+)*,(',,'+-*).,(,.)(.,'+-'(,*'+,'),*),,'+-+(,-(++)(.*')-)',+)*,''-,',-)).+)--').)'+,&¸b  Â‡ŒY _Ñ
§ ÂÿåÆ
"ó‡ <  ,$
×Öwu    .Í·‘2  !ò·   ,ìXç    1†ë   ý¥¡ B † Õ’=      cƒŸB3
"l¡ønÔ}  ÂÐ

9ðéS      ö äý>>
r
¡sŒÕæ      Ñ
l݉:Ï, ¼  §s
$/<”1Xhž      x
ËL|Ÿ¶ à%„f«­      ¬uG[¯ŸÀ?t¢+ Óão0Tö”ÿ      Í©D~ößP^ª˜q\?‰Ø=´Y£é…îòMÞÑÏz4Ð2FMMxDI,.Ç! ¸E£v—.Óñ ¹      ¨ÀG?™J“ TËÃÁ.?žÝi¢%Þ°ÀQø¹Òz#ñ2ˆKl˜Jœ:ÆË®ûB)ÀÃàü?m'ªC/ÜÔbʊ摵ù¥ŸÅ\‹)§p‡Ú“ÊՋ¸JÎàaºÀz\¸ R’öÿ—ø‹`ÏzîÒCØé?XpÛr„ 0ÀòqÀ§„(Û
“  _–uŽƒæM:ÇTTÀÞ‰’äM±³aCL"ÛÄSG§s :&
]¥Î™;&QŸŠRG¬ˆ!įë4T:?šäÏ–J,O~3)\Œ—s¹À      &ëë4™àËkGXŒnD&EpI ðï +/aÀ©ÃÉ´Ñð¯f]Œœm\ù7ri=S¾ñ Ë~œ×Ù½°¤¢­³c!Jipn‹à5ÔÌÔâã̍šÒõÔžz?\š³©´ Txxa+ ÚÞà "̲ÒòL²Ÿvgc|€`<%<VO& #RZYOs§À²“od~Œm^jsuZ72]x„}œ·ÞöÑ¥‘š¨©’…ƒ«ÌÀ¦~py˜¤¡­Ì.ôéâëàÏÁÔéþúãÑÇØåçâî0MQE.+3. 
â×âëþ
/NND0&!3?/#'÷ÔÉÝû;Ypd?$"&
 àÅÇâôû 8`r`=%"íæø
ò 1         V
=====End of Sample====

This sample contains a header (plain text) plus one section of pseudobinary A/D converter output data.
0
 
LVL 2

Expert Comment

by:alexstewart@beta
ID: 2650204
If the 'text' part is always the same length then

MyStream.read(MyString,StringLength);

if not then you have to read in and look for the end of the line (see below)...

After that your line seems to have space characters as separators, so
you can pull the words off one at a time with a function like this...

function  StripWord(sx:string; count:word=1):string;
var
   n,px:integer;
begin
for n:=1 to count do begin
    sx:=trimleft(sx);
    px:=pos(' ',sx);
    if px>0 then begin
       delete(sx,1,px);
       end
     else sx:='';
    end;
result:=sx;
end;



function gettextline( MyStream:TStream; var sx:string):Integer;
const
    MaxLen=1024*10;
//in case of a binary file-give up
var
    err,nb,nx:integer;
    ch:char;
begin
err:=0;
nx:=0;
setlength(sx,MaxLen);
nb:=MyStream.read(ch,1);
if nb<1 then err:=1;//at eof
while (nb=1) do begin
      case ch of
           #10:break;
           #13:;
           else begin
                inc(nx);
                sx[nx]:=ch;
                end;
           end;
      if nx>=MaxLen then begin
         err:=2;//no linefeed found-give up
         break;
         end;
      nb:=MyStream.read(ch,1);
      end;
setlength(sx,nx);
result:=nx;
if err<>0 then result:=-err;
end;

TFileStream is the stream type for files, here is how I use it
....
var stf:TFileStream;
....
try
stf:=TFileStream.create(fn,fmopenread or fmShareDenyNone);
except
on exception do begin
   showmessage('problem opening file '+fn);
   exit;
   end;
end;
try
stf.read(Data,nBytes);
finally
  stf.free;
  end;
....

You could look at the file twice,
open once as a text file, get your numbers, and then open again as a stream or file of byte and read the data.

There are ways to make a 'TextFile' out of a string, so you can use read/readln.

A
0
 
LVL 5

Expert Comment

by:Gwena
ID: 2651365
listening :-)
0
 
LVL 5

Expert Comment

by:Gwena
ID: 2651393
listening :-)
0
 
LVL 2

Author Comment

by:ivobauer
ID: 2660394
I tried up your solution and slightly modified it. It works well so I will accept your answer and points go to your account. Thanks to all!
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

Suggested Solutions

This article explains how to create forms/units independent of other forms/units object names in a delphi project. Have you ever created a form for user input in a Delphi project and then had the need to have that same form in a other Delphi proj…
In this tutorial I will show you how to use the Windows Speech API in Delphi. I will only cover basic functions such as text to speech and controlling the speed of the speech. SAPI Installation First you need to install the SAPI type library, th…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now