Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

stringtokenizer in Delphi/Object Pascal

Posted on 2001-06-05
15
Medium Priority
?
1,402 Views
Last Modified: 2012-06-22
hi there,

I was wondering if there is a sort of StringTokenizer in Delphi. I know that Java has a StringTokenizer, but I can't find it in Delphi, can anyone help me?

A SringTokenizer is an object which divides a string into parts. You can give up a delimiter like:

String deString = "hi,there,this,are,commas";
StringTokenizer someName = new StringTokenizer(deString, ",");

And if i want the next token/element i simply do:
someName.nexttoken();

But this is JAVA, but to implement this in Delphi? Plz hELP!!!

Regards and thanks in advance,
Ruben Bouman,
The Netherlands
0
Comment
Question by:rubenbouman
  • 5
  • 3
  • 2
  • +4
15 Comments
 
LVL 13

Expert Comment

by:Epsylon
ID: 6156387
Check out TStringList and its CommaText property....
0
 
LVL 13

Expert Comment

by:Epsylon
ID: 6156454
Here is an example:

var s: String;
    sl: TStringList;
    i: Integer;
begin
  sl := TStringList.Create;
  s := 'hi,there,this,are,commas';
  sl.CommaText := s;
  for i := 0 to sl.Count -1 do
    ShowMessage(sl[i]);
  sl.Free;
end;
0
 
LVL 3

Expert Comment

by:VSF
ID: 6156596
This is a simple example...
Add:
1 Editbox - Tedit
1 Button - TButton
1 Listbox - TlistBox

//Add this procedure
function sBreakApart(BaseString, BreakString: string; StringList: TStringList): TStringList;
var
EndOfCurrentString: byte;
TempStr: string;
begin
repeat
EndOfCurrentString := Pos(BreakString, BaseString);
if EndOfCurrentString = 0 then
StringList.add(BaseString)
else
StringList.add(Copy(BaseString, 1, EndOfCurrentString - 1));
BaseString := Copy(BaseString, EndOfCurrentString + length(BreakString), length(BaseString) - EndOfCurrentString);

until EndOfCurrentString = 0;
result := StringList;
end;

//Add this code to the OnClick event of the button
procedure TForm1.Button1Click(Sender: TObject);
var
t: TStringList;
begin
t := TStringList.create;
ListBox1.Items.Assign(sBreakApart(Edit1.Text, ' ', t));
t.free;
end;

Run your program and type your name in the Editbox
Press the button
and the list box should show the your name Tokenized using the blank spaces as the string separator.

Adjust this function to any string u want!
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 3

Expert Comment

by:VSF
ID: 6156610
To adjust the function just send to the
BreakString: string
any separator u want!

Hope this helps!

E-mail me if u want a compiled sample or leave your e-mail and I'll send it to you!

VSF
www.enge.cjb.net
www.victory.hpg.com.br
UIN:14016999
0
 
LVL 8

Expert Comment

by:Cesario
ID: 6156761
Hi rubenbouman,

you can use the TPARSER-Object from Delphi. You will find it in the CLASSES Unit.

Best Regards

Cesario
0
 
LVL 26

Accepted Solution

by:
Tomas Helgi Johannsson earned 400 total points
ID: 6156833
Hi!

You can create your own Tokenizer easelly. This is how I did it (coded in 3 minutes in D5 ;)


unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
  TMyClass = class
  private
    TokList: TStrings;
  public
    constructor Create(const s, t: string);
    destructor Destroy;
    procedure AddLists(s: TStrings);
  published
    property thelist: TStrings read TokList write AddLists;
  end;

type
  TForm1 = class(TForm)
    ListBox1: TListBox;
    Edit1: TEdit;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.DFM}

{ TMyClass }

procedure TMyClass.AddLists(s: TStrings);
begin
  TokList.Assign(s);
end;

constructor TMyClass.Create(const s, t: string);
var
  i, j: Integer;
  temp : String;
begin
  temp := s;
  TokList := TStringList.Create;
  i := 1;
  j := 1;
  if Pos(t, temp) > 0 then
    while Pos(t, temp) > 0 do
    begin
      i := Pos(t, temp);
      Toklist.add(Copy(temp, j, i - j));
      delete(temp,j,i);
    end;
    Toklist.Add(Copy(temp, j, i - j));
end;

destructor TMyClass.Destroy;
begin
  TokList.Free;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  myTokl: TMyClass;
begin
  myTokl := TMyClass.Create(Edit1.Text, '/');
  ListBox1.Items.Assign(myTokl.thelist);
  myTokl.Free;
end;

Regards,
Tomas Helgi
0
 
LVL 26

Expert Comment

by:Tomas Helgi Johannsson
ID: 6156859
Hi again!

I forgot to put in the "NextToken" function.
In TMyClass
under private write:
ix : Integer;
under public write :

function NextToken : String;

and the implementation is

function NextToken : String;
begin
    if ix < TokList.Count then
    begin
       result := TokList[ix];
       Inc(ix);
    end
    else
      result := '';
end;

add to the Constructor

ix := 0;

To prevent the user of this class to access the TokList outside the class you simply delete the property line. ;)

Regards,
Tomas Helgi
0
 
LVL 3

Expert Comment

by:VSF
ID: 6157132
No need to post answers! : )

Just lock his question!

VSF
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6157305
//this simple function, does what you need
//it returns the first part as result, and shorten the given string
//for next item simple call again
//if there are no items, then the given string becomes empty
function SplitStr(Var AString : String; ADelim : String) : String;
var APos : Integer;
begin
  Result := '';
  APos := Pos(ADelim,AString);
  if APos > 0 then
  begin
    Result := Copy(AString,1,APos-1);
    AString := copy(Astring,APos+length(Adelim),MaxLongInt);
  end
  else
  begin
    Result := AString;
    AString := '';
  end;
end;

//a usage sample
procedure TForm1.Button1Click(Sender: TObject);
var s : String;
begin
 ListBox1.Clear;
 s := edit1.text;
 while s <> '' do
   listbox1.items.add(SplitStr(s,edit2.text));
end;

meikl ;-)
0
 
LVL 3

Expert Comment

by:VSF
ID: 6157763
so rubenbouman... have u tested any code?!
Feel free to ask if u have any doubts!
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6158777
hi vsf,

who are you, the boy or the girl ?  :-))

sorry to be off topic

meikl ;-)
0
 
LVL 17

Expert Comment

by:geobul
ID: 6158801
Hi,

I've written a small piece of code about dividing a sentence into words. The sentence has to be entered
in the Edit1. Pressing the button passes it as a string to the Parser function and the result is returned
in SList. You can add additional symbols as word delimiters (space, tab and comma here) if you want.
A symbol for the end of the sentence is not mandatory because it ends at the end of the passed string
by default. If such a symbol exists, the rest of the string will be ignored.

unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
 StdCtrls;

type
 TForm1 = class(TForm)
   Button1: TButton;
   Edit1: TEdit;
   Memo1: TMemo;
   procedure Button1Click(Sender: TObject);
   procedure FormCreate(Sender: TObject);
   procedure FormDestroy(Sender: TObject);
 private
   { Private declarations }
 public
   { Public declarations }
 end;

var
 Form1: TForm1;
 SList : TStringList;

implementation

{$R *.DFM}

procedure Parser(Sent: string);
const
 delimiters = [CHR(32),CHR(8),','];
 FullStops = ['.','!','?'];
var
 ch : char;
 i, cnt : integer;
 token : string;
begin
 SList.Clear;
 cnt := Length(Sent);
 token := '';
 if (cnt > 0) then begin
   for i := 1 to cnt do begin
     // Next character
     ch := Sent[i];
     // new token = word
     if ch in Delimiters then begin
       if length(token) > 0 then SList.Add(token);
       token := '';
     end else begin
       // end of sentence
       if ch in FullStops then begin
         if length(token) > 0 then SList.Add(token);
         token := '';
         break;
       end else begin
         // regular character
         token := token + ch;
       end;
     end;
   end;
   // last token
   if length(token) > 0 then SList.Add(token);
 end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
 i: integer;
begin
 Parser(Edit1.Text);
 // you can analize the words here:
 // for i := 0 to SList.Count - 1 do begin
 //   ...
 // end;  
 // The next line is for visualisation only.
 Memo1.Lines.Assign(SList);
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 SList := TStringList.Create;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 SList.Free;
end;

end.

Regards, Geo
0
 

Author Comment

by:rubenbouman
ID: 6159797
Perfect solution! I already used it many many many times!!!

Regards,
Ruben Bouman
0
 
LVL 3

Expert Comment

by:VSF
ID: 6160647
Hi kretzschmar,

I'm the Boy... the girl is my girlfriend Carol. We're togather for almost 3 years now!


0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6160811
I guessed that, but ... i wasn't sure :-))

nice foto
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Objective: - This article will help user in how to convert their numeric value become words. How to use 1. You can copy this code in your Unit as function 2. than you can perform your function by type this code The Code   (CODE) The Im…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
this video summaries big data hadoop online training demo (http://onlineitguru.com/big-data-hadoop-online-training-placement.html) , and covers basics in big data hadoop .
Want to learn how to record your desktop screen without having to use an outside camera. Click on this video and learn how to use the cool google extension called "Screencastify"! Step 1: Open a new google tab Step 2: Go to the left hand upper corn…
Suggested Courses

916 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question