• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1417
  • Last Modified:

stringtokenizer in Delphi/Object Pascal

hi there,

I was wondering if there is a sort of StringTokenizer in Delphi. I know that Java has a StringTokenizer, but I can't find it in Delphi, can anyone help me?

A SringTokenizer is an object which divides a string into parts. You can give up a delimiter like:

String deString = "hi,there,this,are,commas";
StringTokenizer someName = new StringTokenizer(deString, ",");

And if i want the next token/element i simply do:
someName.nexttoken();

But this is JAVA, but to implement this in Delphi? Plz hELP!!!

Regards and thanks in advance,
Ruben Bouman,
The Netherlands
0
rubenbouman
Asked:
rubenbouman
  • 5
  • 3
  • 2
  • +4
1 Solution
 
EpsylonCommented:
Check out TStringList and its CommaText property....
0
 
EpsylonCommented:
Here is an example:

var s: String;
    sl: TStringList;
    i: Integer;
begin
  sl := TStringList.Create;
  s := 'hi,there,this,are,commas';
  sl.CommaText := s;
  for i := 0 to sl.Count -1 do
    ShowMessage(sl[i]);
  sl.Free;
end;
0
 
VSFCommented:
This is a simple example...
Add:
1 Editbox - Tedit
1 Button - TButton
1 Listbox - TlistBox

//Add this procedure
function sBreakApart(BaseString, BreakString: string; StringList: TStringList): TStringList;
var
EndOfCurrentString: byte;
TempStr: string;
begin
repeat
EndOfCurrentString := Pos(BreakString, BaseString);
if EndOfCurrentString = 0 then
StringList.add(BaseString)
else
StringList.add(Copy(BaseString, 1, EndOfCurrentString - 1));
BaseString := Copy(BaseString, EndOfCurrentString + length(BreakString), length(BaseString) - EndOfCurrentString);

until EndOfCurrentString = 0;
result := StringList;
end;

//Add this code to the OnClick event of the button
procedure TForm1.Button1Click(Sender: TObject);
var
t: TStringList;
begin
t := TStringList.create;
ListBox1.Items.Assign(sBreakApart(Edit1.Text, ' ', t));
t.free;
end;

Run your program and type your name in the Editbox
Press the button
and the list box should show the your name Tokenized using the blank spaces as the string separator.

Adjust this function to any string u want!
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
VSFCommented:
To adjust the function just send to the
BreakString: string
any separator u want!

Hope this helps!

E-mail me if u want a compiled sample or leave your e-mail and I'll send it to you!

VSF
www.enge.cjb.net
www.victory.hpg.com.br
UIN:14016999
0
 
CesarioCommented:
Hi rubenbouman,

you can use the TPARSER-Object from Delphi. You will find it in the CLASSES Unit.

Best Regards

Cesario
0
 
Tomas Helgi JohannssonCommented:
Hi!

You can create your own Tokenizer easelly. This is how I did it (coded in 3 minutes in D5 ;)


unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
  TMyClass = class
  private
    TokList: TStrings;
  public
    constructor Create(const s, t: string);
    destructor Destroy;
    procedure AddLists(s: TStrings);
  published
    property thelist: TStrings read TokList write AddLists;
  end;

type
  TForm1 = class(TForm)
    ListBox1: TListBox;
    Edit1: TEdit;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.DFM}

{ TMyClass }

procedure TMyClass.AddLists(s: TStrings);
begin
  TokList.Assign(s);
end;

constructor TMyClass.Create(const s, t: string);
var
  i, j: Integer;
  temp : String;
begin
  temp := s;
  TokList := TStringList.Create;
  i := 1;
  j := 1;
  if Pos(t, temp) > 0 then
    while Pos(t, temp) > 0 do
    begin
      i := Pos(t, temp);
      Toklist.add(Copy(temp, j, i - j));
      delete(temp,j,i);
    end;
    Toklist.Add(Copy(temp, j, i - j));
end;

destructor TMyClass.Destroy;
begin
  TokList.Free;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  myTokl: TMyClass;
begin
  myTokl := TMyClass.Create(Edit1.Text, '/');
  ListBox1.Items.Assign(myTokl.thelist);
  myTokl.Free;
end;

Regards,
Tomas Helgi
0
 
Tomas Helgi JohannssonCommented:
Hi again!

I forgot to put in the "NextToken" function.
In TMyClass
under private write:
ix : Integer;
under public write :

function NextToken : String;

and the implementation is

function NextToken : String;
begin
    if ix < TokList.Count then
    begin
       result := TokList[ix];
       Inc(ix);
    end
    else
      result := '';
end;

add to the Constructor

ix := 0;

To prevent the user of this class to access the TokList outside the class you simply delete the property line. ;)

Regards,
Tomas Helgi
0
 
VSFCommented:
No need to post answers! : )

Just lock his question!

VSF
0
 
kretzschmarCommented:
//this simple function, does what you need
//it returns the first part as result, and shorten the given string
//for next item simple call again
//if there are no items, then the given string becomes empty
function SplitStr(Var AString : String; ADelim : String) : String;
var APos : Integer;
begin
  Result := '';
  APos := Pos(ADelim,AString);
  if APos > 0 then
  begin
    Result := Copy(AString,1,APos-1);
    AString := copy(Astring,APos+length(Adelim),MaxLongInt);
  end
  else
  begin
    Result := AString;
    AString := '';
  end;
end;

//a usage sample
procedure TForm1.Button1Click(Sender: TObject);
var s : String;
begin
 ListBox1.Clear;
 s := edit1.text;
 while s <> '' do
   listbox1.items.add(SplitStr(s,edit2.text));
end;

meikl ;-)
0
 
VSFCommented:
so rubenbouman... have u tested any code?!
Feel free to ask if u have any doubts!
0
 
kretzschmarCommented:
hi vsf,

who are you, the boy or the girl ?  :-))

sorry to be off topic

meikl ;-)
0
 
geobulCommented:
Hi,

I've written a small piece of code about dividing a sentence into words. The sentence has to be entered
in the Edit1. Pressing the button passes it as a string to the Parser function and the result is returned
in SList. You can add additional symbols as word delimiters (space, tab and comma here) if you want.
A symbol for the end of the sentence is not mandatory because it ends at the end of the passed string
by default. If such a symbol exists, the rest of the string will be ignored.

unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
 StdCtrls;

type
 TForm1 = class(TForm)
   Button1: TButton;
   Edit1: TEdit;
   Memo1: TMemo;
   procedure Button1Click(Sender: TObject);
   procedure FormCreate(Sender: TObject);
   procedure FormDestroy(Sender: TObject);
 private
   { Private declarations }
 public
   { Public declarations }
 end;

var
 Form1: TForm1;
 SList : TStringList;

implementation

{$R *.DFM}

procedure Parser(Sent: string);
const
 delimiters = [CHR(32),CHR(8),','];
 FullStops = ['.','!','?'];
var
 ch : char;
 i, cnt : integer;
 token : string;
begin
 SList.Clear;
 cnt := Length(Sent);
 token := '';
 if (cnt > 0) then begin
   for i := 1 to cnt do begin
     // Next character
     ch := Sent[i];
     // new token = word
     if ch in Delimiters then begin
       if length(token) > 0 then SList.Add(token);
       token := '';
     end else begin
       // end of sentence
       if ch in FullStops then begin
         if length(token) > 0 then SList.Add(token);
         token := '';
         break;
       end else begin
         // regular character
         token := token + ch;
       end;
     end;
   end;
   // last token
   if length(token) > 0 then SList.Add(token);
 end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
 i: integer;
begin
 Parser(Edit1.Text);
 // you can analize the words here:
 // for i := 0 to SList.Count - 1 do begin
 //   ...
 // end;  
 // The next line is for visualisation only.
 Memo1.Lines.Assign(SList);
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 SList := TStringList.Create;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 SList.Free;
end;

end.

Regards, Geo
0
 
rubenboumanAuthor Commented:
Perfect solution! I already used it many many many times!!!

Regards,
Ruben Bouman
0
 
VSFCommented:
Hi kretzschmar,

I'm the Boy... the girl is my girlfriend Carol. We're togather for almost 3 years now!


0
 
kretzschmarCommented:
I guessed that, but ... i wasn't sure :-))

nice foto
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 5
  • 3
  • 2
  • +4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now