Solved

ignore comments in a text file

Posted on 2006-07-17
6
306 Views
Last Modified: 2010-04-05
I have a standard text file containing integers.
The file has no consitant structure, and I would like to put comments in it, preceded by a semicolon.

for example:

2                   ;input units
50 150
50 400
3                   ;process units
200 300
200 375
200 450

I would like to read the integers into an array, but ignore the comments.

Any suggestions appreciated.

JonathanH13
0
Comment
Question by:JonathanH13
6 Comments
 
LVL 26

Accepted Solution

by:
Russell Libby earned 200 total points
Comment Utility
Here is one example using regular expressions. Parser can be downloaded from:

http://users.adelphia.net/~rllibby/downloads/regexprex.zip

This is only one way of doing it, as there are a number of different ways.

Regards,
Russell

---

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  reParse:       TRegExpr;
     reMatches:     TRegExprMatches;
     dwIndex:       Integer;
     szValue:       String;
begin

  reParse:=TRegExpr.Create;;
  try
     reParse.LoadFromFile(FileName);
     reParse.Pattern:=';[^\r\n]+';
     reParse.Replace(' ', True);
     reParse.Pattern:='\d+';
     reMatches:=reParse.MatchAll;
     try
        SetLength(result, reMatches.Count);
        for dwIndex:=0 to Pred(reMatches.Count) do
        begin
           SetString(szValue, PChar(@reParse.Source[reMatches[dwIndex].Position]), reMatches[dwIndex].Length);
           result[dwIndex]:=StrToIntDef(szValue, 0);
        end;
     finally
        reMatches.Free;
     end;
  finally
     reParse.Free;
  end;

end;

var  arrInts:       TIntArray;
     dwIndex:       Integer;
begin

  arrInts:=ParseFileList('c:\file1.txt');
  for dwIndex:=0 to Pred(Length(arrInts)) do
  begin
   // .. do whatever
  end;

end;
0
 
LVL 17

Expert Comment

by:TheRealLoki
Comment Utility
I think this should do everything you need

var
  i, j: integer;
  s, s2: string;
  sl: tstringlist;
begin
  sl := tstringlist.create;
  try
    sl.loadfromfile('somefile.txt');
    for i := 0 to pred(sl.count) do
    begin
      s := trim(sl[i]); // take 1 line
      j := pos(';', s);
      if j <> 0 then
      begin // remove comment
        s := trim(copy(s, 1, j-1));
      end;
      j := pos(' ', s); // start check for space
      while (j > 0) do
      begin
        s2 := copy(s, 1, j-1); // coy this number to s2 so we can work with it
        delete(s, 1, j); // remove this number and the space after it
        memo1.lines.add(strToInt(s2)); // display this number in our memo - this is just as an example
        j := pos(' ', s); // check for another space. ie. another number follows
      end;
      if s <> '' then memo1.lines.add(strtoint(s)); // the last number - dont forget to use this
    end;
  finally
    sl.clear;
    sl.free;
  end;  
0
 
LVL 1

Expert Comment

by:Tueblo
Comment Utility
This should do the job:

TDataArray = array of Integer;

function IsWhitespace(c: Char): Boolean;
begin
  result := (c = #09) or (c = #32); // add more whitespaces if neccessary
end;

function GetAndDelNextLine(var str: String): String;
const cEOL = #13#10; // End Of Line sign
var  p: Integer;
begin
  result := '';
  p := Pos(cEOL, str);
  if (p > 0) then
  begin
    result := Copy(str, 1, p - 1);
    str := Copy(str, p + Length(cEOL), high(Integer));
  end;
end;

function GetAndDelNextWord(var str: String): String;
var e, s : Integer;
begin
  s := 1;
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  e := s;
  while (s <= Length(str)) and (not IsWhitespace(str[s])) do INC(s);
  result := Copy(str, e, s - e);
  while (s <= Length(str)) and (IsWhitespace(str[s])) do INC(s);
  str := Copy(str, s, high(Integer));
end;

procedure ReadStuff(x: String; numbers: TDataArray);
var line : String;
    numberStr : String;
    numberVal : Integer;
begin
  while ( Length(x) > 0 ) do
  begin
    line := GetAndDelNextLine(x);
    repeat
      numberStr := GetAndDelNextWord(line);
      if ( numberStr[1] <> ';' ) then
      begin
        numberVal := StrToIntDef(numberStr, high(Integer));
        if ( numberVal <> high(Integer) ) then
        begin
          SetLength(numbers,  Length(numbers) + 1);
          numbers[Length(numbers) - 1] := numberVal;
        end;
      end
      else
        line := '';
    until ( Length(line) = 0 );
  end;
end;

Regards !
Tueblo
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 26

Expert Comment

by:Russell Libby
Comment Utility

Of course, if raw speed is required, then the following should perform very nicely

Russell

----

type
  TIntArray         =  Array of Integer;

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwError:       Integer;
     szValue:       String;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          if (lpszData^ in ['0'..'9']) then
                          begin
                             // Set anchor
                             lpszAnchor:=lpszData;
                             // Parse number
                             while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                             // Extract string
                             SetString(szValue, lpszAnchor, lpszData-lpszAnchor);
                             // Convert to integer
                             Val(szValue, dwValue, dwError);
                             // Check error
                             if (dwError = 0) then
                                listData.Add(Pointer(dwValue))
                             else
                                listData.Add(Pointer(MaxInt));
                          end
                          // Check for comment
                          else if (lpszData^ = ';') then
                          begin
                             // Parse till end of line
                             while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                             // Consume end of line
                             while (lpszData^ in [#10, #13]) do Inc(lpszData);
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 
LVL 26

Expert Comment

by:Russell Libby
Comment Utility

You can also drop the string - > int conversion and perform it manually. Doing so dropped the time on a 3MB file from 140 ms to an average of 38 ms. That time includes creating the resulting integer array.

Russell

---

function ParseFileList(FileName: String): TIntArray;
var  listData:      TList;
     lpszData:      PChar;
     lpszAnchor:    PChar;
     hFile:         THandle;
     hMapping:      THandle;
     dwSize:        DWORD;
     dwValue:       Integer;
     dwMult:        Integer;
begin

  // Create list
  listData:=TList.Create;

  // Resource protection
  try
     // Open the file for reading
     hFile:=CreateFile(PChar(FileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
     // Check handle
     if (hFile <> INVALID_HANDLE_VALUE) then
     begin
        // Resource protection
        try
           // Get the file size
           dwSize:=GetFileSize(hFile, nil);
           // Create the file mapping
           hMapping:=CreateFileMapping(hFile, nil, PAGE_READONLY or SEC_COMMIT, 0, 0, nil);
           // Check mapping
           if (hMapping > 0) then
           begin
              // Resource protection
              try
                 // Map view of file
                 lpszData:=MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0);
                 // Check data
                 if Assigned(lpszData) then
                 begin
                    // Resource protection
                    try
                       // While not null
                       while (lpszData^ > #0) do
                       begin
                          // Skip until a char we want
                          while not(lpszData^ in [#0, ';', '0'..'9']) do Inc(lpszData);
                          // Check for number
                          case lpszData^ of
                             '0'..'9' :
                             begin
                                // Set anchor
                                lpszAnchor:=lpszData;
                                // Parse number
                                while (lpszData^ in ['0'..'9']) do Inc(lpszData);
                                // Calculate size
                                dwSize:=lpszData - lpszAnchor;
                                // Set anchor for backwards walking
                                Inc(lpszAnchor, Pred(dwSize));
                                dwValue:=0;
                                dwMult:=1;
                                while (dwSize > 0) do
                                begin
                                   Inc(dwValue, (Ord(lpszAnchor^) - 48) * dwMult);
                                   dwMult:=dwMult * 10;
                                   Dec(lpszAnchor);
                                   Dec(dwSize);
                                end;
                                listData.Add(Pointer(dwValue))
                             end;
                             ';'      :
                             begin
                                // Parse till end of line
                                while not(lpszData^ in [#10, #13]) do Inc(lpszData);
                                // Consume end of line
                                while (lpszData^ in [#10, #13]) do Inc(lpszData);
                             end;
                          end;
                       end;
                    finally
                       // Unmap view of file
                       UnmapViewOfFile(lpszData);
                    end;
                 end;
              finally
                 // Close the handle
                 CloseHandle(hMapping);
              end;
           end;
        finally
           // Close the handle
           CloseHandle(hFile);
        end;
     end;
  finally
     // Resource protection
     try
        // Convert list to integer array
        SetLength(result, listData.Count);
        // Move data over
        for dwValue:=0 to Pred(listData.Count) do result[dwValue]:=Integer(listData[dwValue]);
     finally
        // Free list
        listData.Free;
     end;
  end;

end;
0
 

Author Comment

by:JonathanH13
Comment Utility
rllibby - thanks for the elegant solution, it works perfectly, and thanks to others for your contributions.

JonathanH13
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Illustrator's Shape Builder tool will let you combine shapes visually and interactively. This video shows the Mac version, but the tool works the same way in Windows. To follow along with this video, you can draw your own shapes or download the file…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

7 Experts available now in Live!

Get 1:1 Help Now