Solved

ignore comments in a text file

Posted on 2006-07-17
6
307 Views
Last Modified: 2010-04-05
I have a standard text file containing integers.
The file has no consitant structure, and I would like to put comments in it, preceded by a semicolon.

for example:

2                   ;input units
50 150
50 400
3                   ;process units
200 300
200 375
200 450

I would like to read the integers into an array, but ignore the comments.

Any suggestions appreciated.

JonathanH13
0
Comment
Question by:JonathanH13
6 Comments
 
LVL 26

Accepted Solution

by:
Russell Libby earned 200 total points
ID: 17123614
Here is one example using regular expressions. Parser can be downloaded from:

http://users.adelphia.net/~rllibby/downloads/regexprex.zip

This is only one way of doing it, as there are a number of different ways.

Regards,
Russell

---

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  reParse:       TRegExpr;
     reMatches:     TRegExprMatches;
     dwIndex:       Integer;
     szValue:       String;
begin

  reParse:=TRegExpr.Create;;
  try
     reParse.LoadFromFile(FileName);
     reParse.Pattern:=';[^\r\n]+';
     reParse.Replace(' ', True);
     reParse.Pattern:='\d+';
     reMatches:=reParse.MatchAll;
     try
        SetLength(result, reMatches.Count);
        for dwIndex:=0 to Pred(reMatches.Count) do
        begin
           SetString(szValue, PChar(@reParse.Source[reMatches[dwIndex].Position]), reMatches[dwIndex].Length);
           result[dwIndex]:=StrToIntDef(szValue, 0);
        end;
     finally
        reMatches.Free;
     end;
  finally
     reParse.Free;
  end;

end;

var  arrInts:       TIntArray;
     dwIndex:       Integer;
begin

  arrInts:=ParseFileList('c:\file1.txt');
  for dwIndex:=0 to Pred(Length(arrInts)) do
  begin
   // .. do whatever
  end;

end;
0
 
LVL 17

Expert Comment

by:TheRealLoki
ID: 17123654
I think this should do everything you need

var
  i, j: integer;
  s, s2: string;
  sl: tstringlist;
begin
  sl := tstringlist.create;
  try
    sl.loadfromfile('somefile.txt');
    for i := 0 to pred(sl.count) do
    begin
      s := trim(sl[i]); // take 1 line
      j := pos(';', s);
      if j <> 0 then
      begin // remove comment
        s := trim(copy(s, 1, j-1));
      end;
      j := pos(' ', s); // start check for space
      while (j > 0) do
      begin
        s2 := copy(s, 1, j-1); // coy this number to s2 so we can work with it
        delete(s, 1, j); // remove this number and the space after it
        memo1.lines.add(strToInt(s2)); // display this number in our memo - this is just as an example
        j := pos(' ', s); // check for another space. ie. another number follows
      end;
      if s <> '' then memo1.lines.add(strtoint(s)); // the last number - dont forget to use this
    end;
  finally
    sl.clear;
    sl.free;
  end;  
0
 
LVL 1

Expert Comment

by:Tueblo
ID: 17123695
This should do the job:

TDataArray = array of Integer;

function IsWhitespace(c: Char): Boolean;
begin
  result := (c = #09) or (c = #32); // add more whitespaces if neccessary
end;

function GetAndDelNextLine(var str: String): String;
const cEOL = #13#10; // End Of Line sign
var  p: Integer;
begin
  result := '';
  p := Pos(cEOL, str);
  if (p > 0) then
  begin
    result := Copy(str, 1, p - 1);
    str := Copy(str, p + Length(cEOL), high(Integer));
  end;
end;

function GetAndDelNextWord(var str: String): String;
var e, s : Integer;
begin
  s := 1;
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  e := s;
  while (s <= Length(str)) and (not IsWhitespace(str[s])) do INC(s);
  result := Copy(str, e, s - e);
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  str := Copy(str, s, high(Integer));
end;

procedure ReadStuff(x: String; numbers: TDataArray);
var line : String;
    numberStr : String;
    numberVal : Integer;
begin
  while ( Length(x) > 0 ) do
  begin
    line := GetAndDelNextLine(x);
    repeat
      numberStr := GetAndDelNextWord(line);
      if ( numberStr[1] <> ';' ) then
      begin
        numberVal := StrToIntDef(numberStr, high(Integer));
        if ( numberVal <> high(Integer) ) then
        begin
          SetLength(numbers,  Length(numbers) + 1);
          numbers[Length(numbers) - 1] := numberVal;
        end;
      end
      else
        line := '';
    until ( Length(line) = 0 );
  end;
end;

Regards !
Tueblo
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 26

Expert Comment

by:Russell Libby
ID: 17123973

Of course, if raw speed is required, then the following should perform very nicely

Russell

----

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwError:       Integer;
     szValue:       String;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          if (lpszData^ in ['0'..'9']) then
                          begin
                             // Set anchor
                             lpszAnchor:=lpszData;
                             // Parse number
                             while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                             // Extract string
                             SetString(szValue, lpszAnchor, lpszData-lpszAnchor);
                             // Convert to integer
                             Val(szValue, dwValue, dwError);
                             // Check error
                             if (dwError = 0) then
                                listData.Add(Pointer(dwValue))
                             else
                                listData.Add(Pointer(MaxInt));
                          end
                          // Check for comment
                          else if (lpszData^ = ';') then
                          begin
                             // Parse till end of line
                             while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                             // Consume end of line
                             while (lpszData^ in [#10, #13]) do Inc(lpszData);
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17124137

You can also drop the string - > int conversion and perform it manually. Doing so dropped the time on a 3MB file from 140 ms to an average of 38 ms. That time includes creating the resulting integer array.

Russell

---

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwMult:        Integer;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          case lpszData^ of
                             '0'..'9' :
                             begin
                                // Set anchor
                                lpszAnchor:=lpszData;
                                // Parse number
                                while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                                // Calculate size
                                dwSize:=lpszData - lpszAnchor;
                                // Set anchor for backwards walking
                                Inc(lpszAnchor, Pred(dwSize));
                                dwValue:=0;
                                dwMult:=1;
                                while (dwSize > 0) do
                                begin
                                   Inc(dwValue, (Ord(lpszAnchor^) - 48) * dwMult);
                                   dwMult:=dwMult * 10;
                                   Dec(lpszAnchor);
                                   Dec(dwSize);
                                end;
                                listData.Add(Pointer(dwValue))
                             end;
                             ';'      :
                             begin
                                // Parse till end of line
                                while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                                // Consume end of line
                                while (lpszData^ in [#10, #13]) do Inc(lpszData);
                             end;
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 

Author Comment

by:JonathanH13
ID: 17129312
rllibby - thanks for the elegant solution, it works perfectly, and thanks to others for your contributions.

JonathanH13
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Objective: - This article will help user in how to convert their numeric value become words. How to use 1. You can copy this code in your Unit as function 2. than you can perform your function by type this code The Code   (CODE) The Im…
In my programming career I have only very rarely run into situations where operator overloading would be of any use in my work.  Normally those situations involved math with either overly large numbers (hundreds of thousands of digits or accuracy re…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
Need to grow your business through quality cloud solutions? With everything required to build a cloud platform and solution, you may feel like the distance between you and the cloud is quite long. Help is here. Spend some time learning about the Con…

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now