Solved

stringtokenizer in Delphi/Object Pascal

Posted on 2001-06-05
15
1,262 Views
Last Modified: 2012-06-22
hi there,

I was wondering if there is a sort of StringTokenizer in Delphi. I know that Java has a StringTokenizer, but I can't find it in Delphi, can anyone help me?

A SringTokenizer is an object which divides a string into parts. You can give up a delimiter like:

String deString = "hi,there,this,are,commas";
StringTokenizer someName = new StringTokenizer(deString, ",");

And if i want the next token/element i simply do:
someName.nexttoken();

But this is JAVA, but to implement this in Delphi? Plz hELP!!!

Regards and thanks in advance,
Ruben Bouman,
The Netherlands
0
Comment
Question by:rubenbouman
  • 5
  • 3
  • 2
  • +4
15 Comments
 
LVL 13

Expert Comment

by:Epsylon
ID: 6156387
Check out TStringList and its CommaText property....
0
 
LVL 13

Expert Comment

by:Epsylon
ID: 6156454
Here is an example:

var s: String;
    sl: TStringList;
    i: Integer;
begin
  sl := TStringList.Create;
  s := 'hi,there,this,are,commas';
  sl.CommaText := s;
  for i := 0 to sl.Count -1 do
    ShowMessage(sl[i]);
  sl.Free;
end;
0
 
LVL 3

Expert Comment

by:VSF
ID: 6156596
This is a simple example...
Add:
1 Editbox - Tedit
1 Button - TButton
1 Listbox - TlistBox

//Add this procedure
function sBreakApart(BaseString, BreakString: string; StringList: TStringList): TStringList;
var
EndOfCurrentString: byte;
TempStr: string;
begin
repeat
EndOfCurrentString := Pos(BreakString, BaseString);
if EndOfCurrentString = 0 then
StringList.add(BaseString)
else
StringList.add(Copy(BaseString, 1, EndOfCurrentString - 1));
BaseString := Copy(BaseString, EndOfCurrentString + length(BreakString), length(BaseString) - EndOfCurrentString);

until EndOfCurrentString = 0;
result := StringList;
end;

//Add this code to the OnClick event of the button
procedure TForm1.Button1Click(Sender: TObject);
var
t: TStringList;
begin
t := TStringList.create;
ListBox1.Items.Assign(sBreakApart(Edit1.Text, ' ', t));
t.free;
end;

Run your program and type your name in the Editbox
Press the button
and the list box should show the your name Tokenized using the blank spaces as the string separator.

Adjust this function to any string u want!
0
 
LVL 3

Expert Comment

by:VSF
ID: 6156610
To adjust the function just send to the
BreakString: string
any separator u want!

Hope this helps!

E-mail me if u want a compiled sample or leave your e-mail and I'll send it to you!

VSF
www.enge.cjb.net
www.victory.hpg.com.br
UIN:14016999
0
 
LVL 8

Expert Comment

by:Cesario
ID: 6156761
Hi rubenbouman,

you can use the TPARSER-Object from Delphi. You will find it in the CLASSES Unit.

Best Regards

Cesario
0
 
LVL 24

Accepted Solution

by:
Tomas Helgi Johannsson earned 100 total points
ID: 6156833
Hi!

You can create your own Tokenizer easelly. This is how I did it (coded in 3 minutes in D5 ;)


unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
  TMyClass = class
  private
    TokList: TStrings;
  public
    constructor Create(const s, t: string);
    destructor Destroy;
    procedure AddLists(s: TStrings);
  published
    property thelist: TStrings read TokList write AddLists;
  end;

type
  TForm1 = class(TForm)
    ListBox1: TListBox;
    Edit1: TEdit;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.DFM}

{ TMyClass }

procedure TMyClass.AddLists(s: TStrings);
begin
  TokList.Assign(s);
end;

constructor TMyClass.Create(const s, t: string);
var
  i, j: Integer;
  temp : String;
begin
  temp := s;
  TokList := TStringList.Create;
  i := 1;
  j := 1;
  if Pos(t, temp) > 0 then
    while Pos(t, temp) > 0 do
    begin
      i := Pos(t, temp);
      Toklist.add(Copy(temp, j, i - j));
      delete(temp,j,i);
    end;
    Toklist.Add(Copy(temp, j, i - j));
end;

destructor TMyClass.Destroy;
begin
  TokList.Free;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  myTokl: TMyClass;
begin
  myTokl := TMyClass.Create(Edit1.Text, '/');
  ListBox1.Items.Assign(myTokl.thelist);
  myTokl.Free;
end;

Regards,
Tomas Helgi
0
 
LVL 24

Expert Comment

by:Tomas Helgi Johannsson
ID: 6156859
Hi again!

I forgot to put in the "NextToken" function.
In TMyClass
under private write:
ix : Integer;
under public write :

function NextToken : String;

and the implementation is

function NextToken : String;
begin
    if ix < TokList.Count then
    begin
       result := TokList[ix];
       Inc(ix);
    end
    else
      result := '';
end;

add to the Constructor

ix := 0;

To prevent the user of this class to access the TokList outside the class you simply delete the property line. ;)

Regards,
Tomas Helgi
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 3

Expert Comment

by:VSF
ID: 6157132
No need to post answers! : )

Just lock his question!

VSF
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6157305
//this simple function, does what you need
//it returns the first part as result, and shorten the given string
//for next item simple call again
//if there are no items, then the given string becomes empty
function SplitStr(Var AString : String; ADelim : String) : String;
var APos : Integer;
begin
  Result := '';
  APos := Pos(ADelim,AString);
  if APos > 0 then
  begin
    Result := Copy(AString,1,APos-1);
    AString := copy(Astring,APos+length(Adelim),MaxLongInt);
  end
  else
  begin
    Result := AString;
    AString := '';
  end;
end;

//a usage sample
procedure TForm1.Button1Click(Sender: TObject);
var s : String;
begin
 ListBox1.Clear;
 s := edit1.text;
 while s <> '' do
   listbox1.items.add(SplitStr(s,edit2.text));
end;

meikl ;-)
0
 
LVL 3

Expert Comment

by:VSF
ID: 6157763
so rubenbouman... have u tested any code?!
Feel free to ask if u have any doubts!
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6158777
hi vsf,

who are you, the boy or the girl ?  :-))

sorry to be off topic

meikl ;-)
0
 
LVL 17

Expert Comment

by:geobul
ID: 6158801
Hi,

I've written a small piece of code about dividing a sentence into words. The sentence has to be entered
in the Edit1. Pressing the button passes it as a string to the Parser function and the result is returned
in SList. You can add additional symbols as word delimiters (space, tab and comma here) if you want.
A symbol for the end of the sentence is not mandatory because it ends at the end of the passed string
by default. If such a symbol exists, the rest of the string will be ignored.

unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
 StdCtrls;

type
 TForm1 = class(TForm)
   Button1: TButton;
   Edit1: TEdit;
   Memo1: TMemo;
   procedure Button1Click(Sender: TObject);
   procedure FormCreate(Sender: TObject);
   procedure FormDestroy(Sender: TObject);
 private
   { Private declarations }
 public
   { Public declarations }
 end;

var
 Form1: TForm1;
 SList : TStringList;

implementation

{$R *.DFM}

procedure Parser(Sent: string);
const
 delimiters = [CHR(32),CHR(8),','];
 FullStops = ['.','!','?'];
var
 ch : char;
 i, cnt : integer;
 token : string;
begin
 SList.Clear;
 cnt := Length(Sent);
 token := '';
 if (cnt > 0) then begin
   for i := 1 to cnt do begin
     // Next character
     ch := Sent[i];
     // new token = word
     if ch in Delimiters then begin
       if length(token) > 0 then SList.Add(token);
       token := '';
     end else begin
       // end of sentence
       if ch in FullStops then begin
         if length(token) > 0 then SList.Add(token);
         token := '';
         break;
       end else begin
         // regular character
         token := token + ch;
       end;
     end;
   end;
   // last token
   if length(token) > 0 then SList.Add(token);
 end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
 i: integer;
begin
 Parser(Edit1.Text);
 // you can analize the words here:
 // for i := 0 to SList.Count - 1 do begin
 //   ...
 // end;  
 // The next line is for visualisation only.
 Memo1.Lines.Assign(SList);
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 SList := TStringList.Create;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 SList.Free;
end;

end.

Regards, Geo
0
 

Author Comment

by:rubenbouman
ID: 6159797
Perfect solution! I already used it many many many times!!!

Regards,
Ruben Bouman
0
 
LVL 3

Expert Comment

by:VSF
ID: 6160647
Hi kretzschmar,

I'm the Boy... the girl is my girlfriend Carol. We're togather for almost 3 years now!


0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6160811
I guessed that, but ... i wasn't sure :-))

nice foto
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
This video discusses moving either the default database or any database to a new volume.
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now