?
Solved

ignore comments in a text file

Posted on 2006-07-17
6
Medium Priority
?
321 Views
Last Modified: 2010-04-05
I have a standard text file containing integers.
The file has no consitant structure, and I would like to put comments in it, preceded by a semicolon.

for example:

2                   ;input units
50 150
50 400
3                   ;process units
200 300
200 375
200 450

I would like to read the integers into an array, but ignore the comments.

Any suggestions appreciated.

JonathanH13
0
Comment
Question by:JonathanH13
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 26

Accepted Solution

by:
Russell Libby earned 800 total points
ID: 17123614
Here is one example using regular expressions. Parser can be downloaded from:

http://users.adelphia.net/~rllibby/downloads/regexprex.zip

This is only one way of doing it, as there are a number of different ways.

Regards,
Russell

---

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  reParse:       TRegExpr;
     reMatches:     TRegExprMatches;
     dwIndex:       Integer;
     szValue:       String;
begin

  reParse:=TRegExpr.Create;;
  try
     reParse.LoadFromFile(FileName);
     reParse.Pattern:=';[^\r\n]+';
     reParse.Replace(' ', True);
     reParse.Pattern:='\d+';
     reMatches:=reParse.MatchAll;
     try
        SetLength(result, reMatches.Count);
        for dwIndex:=0 to Pred(reMatches.Count) do
        begin
           SetString(szValue, PChar(@reParse.Source[reMatches[dwIndex].Position]), reMatches[dwIndex].Length);
           result[dwIndex]:=StrToIntDef(szValue, 0);
        end;
     finally
        reMatches.Free;
     end;
  finally
     reParse.Free;
  end;

end;

var  arrInts:       TIntArray;
     dwIndex:       Integer;
begin

  arrInts:=ParseFileList('c:\file1.txt');
  for dwIndex:=0 to Pred(Length(arrInts)) do
  begin
   // .. do whatever
  end;

end;
0
 
LVL 17

Expert Comment

by:TheRealLoki
ID: 17123654
I think this should do everything you need

var
  i, j: integer;
  s, s2: string;
  sl: tstringlist;
begin
  sl := tstringlist.create;
  try
    sl.loadfromfile('somefile.txt');
    for i := 0 to pred(sl.count) do
    begin
      s := trim(sl[i]); // take 1 line
      j := pos(';', s);
      if j <> 0 then
      begin // remove comment
        s := trim(copy(s, 1, j-1));
      end;
      j := pos(' ', s); // start check for space
      while (j > 0) do
      begin
        s2 := copy(s, 1, j-1); // coy this number to s2 so we can work with it
        delete(s, 1, j); // remove this number and the space after it
        memo1.lines.add(strToInt(s2)); // display this number in our memo - this is just as an example
        j := pos(' ', s); // check for another space. ie. another number follows
      end;
      if s <> '' then memo1.lines.add(strtoint(s)); // the last number - dont forget to use this
    end;
  finally
    sl.clear;
    sl.free;
  end;  
0
 
LVL 1

Expert Comment

by:Tueblo
ID: 17123695
This should do the job:

TDataArray = array of Integer;

function IsWhitespace(c: Char): Boolean;
begin
  result := (c = #09) or (c = #32); // add more whitespaces if neccessary
end;

function GetAndDelNextLine(var str: String): String;
const cEOL = #13#10; // End Of Line sign
var  p: Integer;
begin
  result := '';
  p := Pos(cEOL, str);
  if (p > 0) then
  begin
    result := Copy(str, 1, p - 1);
    str := Copy(str, p + Length(cEOL), high(Integer));
  end;
end;

function GetAndDelNextWord(var str: String): String;
var e, s : Integer;
begin
  s := 1;
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  e := s;
  while (s <= Length(str)) and (not IsWhitespace(str[s])) do INC(s);
  result := Copy(str, e, s - e);
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  str := Copy(str, s, high(Integer));
end;

procedure ReadStuff(x: String; numbers: TDataArray);
var line : String;
    numberStr : String;
    numberVal : Integer;
begin
  while ( Length(x) > 0 ) do
  begin
    line := GetAndDelNextLine(x);
    repeat
      numberStr := GetAndDelNextWord(line);
      if ( numberStr[1] <> ';' ) then
      begin
        numberVal := StrToIntDef(numberStr, high(Integer));
        if ( numberVal <> high(Integer) ) then
        begin
          SetLength(numbers,  Length(numbers) + 1);
          numbers[Length(numbers) - 1] := numberVal;
        end;
      end
      else
        line := '';
    until ( Length(line) = 0 );
  end;
end;

Regards !
Tueblo
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 26

Expert Comment

by:Russell Libby
ID: 17123973

Of course, if raw speed is required, then the following should perform very nicely

Russell

----

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwError:       Integer;
     szValue:       String;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          if (lpszData^ in ['0'..'9']) then
                          begin
                             // Set anchor
                             lpszAnchor:=lpszData;
                             // Parse number
                             while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                             // Extract string
                             SetString(szValue, lpszAnchor, lpszData-lpszAnchor);
                             // Convert to integer
                             Val(szValue, dwValue, dwError);
                             // Check error
                             if (dwError = 0) then
                                listData.Add(Pointer(dwValue))
                             else
                                listData.Add(Pointer(MaxInt));
                          end
                          // Check for comment
                          else if (lpszData^ = ';') then
                          begin
                             // Parse till end of line
                             while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                             // Consume end of line
                             while (lpszData^ in [#10, #13]) do Inc(lpszData);
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17124137

You can also drop the string - > int conversion and perform it manually. Doing so dropped the time on a 3MB file from 140 ms to an average of 38 ms. That time includes creating the resulting integer array.

Russell

---

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwMult:        Integer;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          case lpszData^ of
                             '0'..'9' :
                             begin
                                // Set anchor
                                lpszAnchor:=lpszData;
                                // Parse number
                                while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                                // Calculate size
                                dwSize:=lpszData - lpszAnchor;
                                // Set anchor for backwards walking
                                Inc(lpszAnchor, Pred(dwSize));
                                dwValue:=0;
                                dwMult:=1;
                                while (dwSize > 0) do
                                begin
                                   Inc(dwValue, (Ord(lpszAnchor^) - 48) * dwMult);
                                   dwMult:=dwMult * 10;
                                   Dec(lpszAnchor);
                                   Dec(dwSize);
                                end;
                                listData.Add(Pointer(dwValue))
                             end;
                             ';'      :
                             begin
                                // Parse till end of line
                                while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                                // Consume end of line
                                while (lpszData^ in [#10, #13]) do Inc(lpszData);
                             end;
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 

Author Comment

by:JonathanH13
ID: 17129312
rllibby - thanks for the elegant solution, it works perfectly, and thanks to others for your contributions.

JonathanH13
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The uses clause is one of those things that just tends to grow and grow. Most of the time this is in the main form, as it's from this form that all others are called. If you have a big application (including many forms), the uses clause in the in…
Creating an auto free TStringList The TStringList is a basic and frequently used object in Delphi. On many occasions, you may want to create a temporary list, process some items in the list and be done with the list. In such cases, you have to…
If you’ve ever visited a web page and noticed a cool font that you really liked the look of, but couldn’t figure out which font it was so that you could use it for your own work, then this video is for you! In this Micro Tutorial, you'll learn yo…
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…
Suggested Courses
Course of the Month11 days, 15 hours left to enroll

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question