Solved

ignore comments in a text file

Posted on 2006-07-17
6
313 Views
Last Modified: 2010-04-05
I have a standard text file containing integers.
The file has no consitant structure, and I would like to put comments in it, preceded by a semicolon.

for example:

2                   ;input units
50 150
50 400
3                   ;process units
200 300
200 375
200 450

I would like to read the integers into an array, but ignore the comments.

Any suggestions appreciated.

JonathanH13
0
Comment
Question by:JonathanH13
6 Comments
 
LVL 26

Accepted Solution

by:
Russell Libby earned 200 total points
ID: 17123614
Here is one example using regular expressions. Parser can be downloaded from:

http://users.adelphia.net/~rllibby/downloads/regexprex.zip

This is only one way of doing it, as there are a number of different ways.

Regards,
Russell

---

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  reParse:       TRegExpr;
     reMatches:     TRegExprMatches;
     dwIndex:       Integer;
     szValue:       String;
begin

  reParse:=TRegExpr.Create;;
  try
     reParse.LoadFromFile(FileName);
     reParse.Pattern:=';[^\r\n]+';
     reParse.Replace(' ', True);
     reParse.Pattern:='\d+';
     reMatches:=reParse.MatchAll;
     try
        SetLength(result, reMatches.Count);
        for dwIndex:=0 to Pred(reMatches.Count) do
        begin
           SetString(szValue, PChar(@reParse.Source[reMatches[dwIndex].Position]), reMatches[dwIndex].Length);
           result[dwIndex]:=StrToIntDef(szValue, 0);
        end;
     finally
        reMatches.Free;
     end;
  finally
     reParse.Free;
  end;

end;

var  arrInts:       TIntArray;
     dwIndex:       Integer;
begin

  arrInts:=ParseFileList('c:\file1.txt');
  for dwIndex:=0 to Pred(Length(arrInts)) do
  begin
   // .. do whatever
  end;

end;
0
 
LVL 17

Expert Comment

by:TheRealLoki
ID: 17123654
I think this should do everything you need

var
  i, j: integer;
  s, s2: string;
  sl: tstringlist;
begin
  sl := tstringlist.create;
  try
    sl.loadfromfile('somefile.txt');
    for i := 0 to pred(sl.count) do
    begin
      s := trim(sl[i]); // take 1 line
      j := pos(';', s);
      if j <> 0 then
      begin // remove comment
        s := trim(copy(s, 1, j-1));
      end;
      j := pos(' ', s); // start check for space
      while (j > 0) do
      begin
        s2 := copy(s, 1, j-1); // coy this number to s2 so we can work with it
        delete(s, 1, j); // remove this number and the space after it
        memo1.lines.add(strToInt(s2)); // display this number in our memo - this is just as an example
        j := pos(' ', s); // check for another space. ie. another number follows
      end;
      if s <> '' then memo1.lines.add(strtoint(s)); // the last number - dont forget to use this
    end;
  finally
    sl.clear;
    sl.free;
  end;  
0
 
LVL 1

Expert Comment

by:Tueblo
ID: 17123695
This should do the job:

TDataArray = array of Integer;

function IsWhitespace(c: Char): Boolean;
begin
  result := (c = #09) or (c = #32); // add more whitespaces if neccessary
end;

function GetAndDelNextLine(var str: String): String;
const cEOL = #13#10; // End Of Line sign
var  p: Integer;
begin
  result := '';
  p := Pos(cEOL, str);
  if (p > 0) then
  begin
    result := Copy(str, 1, p - 1);
    str := Copy(str, p + Length(cEOL), high(Integer));
  end;
end;

function GetAndDelNextWord(var str: String): String;
var e, s : Integer;
begin
  s := 1;
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  e := s;
  while (s <= Length(str)) and (not IsWhitespace(str[s])) do INC(s);
  result := Copy(str, e, s - e);
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  str := Copy(str, s, high(Integer));
end;

procedure ReadStuff(x: String; numbers: TDataArray);
var line : String;
    numberStr : String;
    numberVal : Integer;
begin
  while ( Length(x) > 0 ) do
  begin
    line := GetAndDelNextLine(x);
    repeat
      numberStr := GetAndDelNextWord(line);
      if ( numberStr[1] <> ';' ) then
      begin
        numberVal := StrToIntDef(numberStr, high(Integer));
        if ( numberVal <> high(Integer) ) then
        begin
          SetLength(numbers,  Length(numbers) + 1);
          numbers[Length(numbers) - 1] := numberVal;
        end;
      end
      else
        line := '';
    until ( Length(line) = 0 );
  end;
end;

Regards !
Tueblo
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 26

Expert Comment

by:Russell Libby
ID: 17123973

Of course, if raw speed is required, then the following should perform very nicely

Russell

----

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwError:       Integer;
     szValue:       String;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          if (lpszData^ in ['0'..'9']) then
                          begin
                             // Set anchor
                             lpszAnchor:=lpszData;
                             // Parse number
                             while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                             // Extract string
                             SetString(szValue, lpszAnchor, lpszData-lpszAnchor);
                             // Convert to integer
                             Val(szValue, dwValue, dwError);
                             // Check error
                             if (dwError = 0) then
                                listData.Add(Pointer(dwValue))
                             else
                                listData.Add(Pointer(MaxInt));
                          end
                          // Check for comment
                          else if (lpszData^ = ';') then
                          begin
                             // Parse till end of line
                             while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                             // Consume end of line
                             while (lpszData^ in [#10, #13]) do Inc(lpszData);
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17124137

You can also drop the string - > int conversion and perform it manually. Doing so dropped the time on a 3MB file from 140 ms to an average of 38 ms. That time includes creating the resulting integer array.

Russell

---

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwMult:        Integer;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          case lpszData^ of
                             '0'..'9' :
                             begin
                                // Set anchor
                                lpszAnchor:=lpszData;
                                // Parse number
                                while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                                // Calculate size
                                dwSize:=lpszData - lpszAnchor;
                                // Set anchor for backwards walking
                                Inc(lpszAnchor, Pred(dwSize));
                                dwValue:=0;
                                dwMult:=1;
                                while (dwSize > 0) do
                                begin
                                   Inc(dwValue, (Ord(lpszAnchor^) - 48) * dwMult);
                                   dwMult:=dwMult * 10;
                                   Dec(lpszAnchor);
                                   Dec(dwSize);
                                end;
                                listData.Add(Pointer(dwValue))
                             end;
                             ';'      :
                             begin
                                // Parse till end of line
                                while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                                // Consume end of line
                                while (lpszData^ in [#10, #13]) do Inc(lpszData);
                             end;
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 

Author Comment

by:JonathanH13
ID: 17129312
rllibby - thanks for the elegant solution, it works perfectly, and thanks to others for your contributions.

JonathanH13
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
In this tutorial I will show you how to use the Windows Speech API in Delphi. I will only cover basic functions such as text to speech and controlling the speed of the speech. SAPI Installation First you need to install the SAPI type library, th…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question