Link to home
Start Free TrialLog in
Avatar of zizi21
zizi21

asked on

urgent: tokenizer - pls help

hi,

i am reading the file and when there is too much of white space, it is taking a long time...

is this the correct way to tokenize ?

tok = strtok(string, DELIMITER);
         while(tok!=NULL)
         {
           
               printf("%s\n",tok);

             tok = strtok(NULL, LIMIT);
         }

i thought, when we tokenize it we remove blank space, empty lines ,

this is my LIMIT =" \n\t\r"

could anyone pls help? thanks
ASKER CERTIFIED SOLUTION
Avatar of Infinity08
Infinity08
Flag of Belgium image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of zizi21
zizi21

ASKER

at times, the program would read blank lines from file and it is stopping there.....
Avatar of zizi21

ASKER

it could be any string...
Avatar of zizi21

ASKER

i need to take strings from file and tokenize it by empty space..so, i added /n/t/r..is there any other tabs or special characters that i could add?
I'm sorry. I need to go now - someone else will probably step in ;) But, can you give an example of where it goes wrong ? ie. An input string that gives wrong output ?
Avatar of zizi21

ASKER

ok
[ some text ] [ some text ]  [some text]
empty line and space
empty line and space...


Avatar of zizi21

ASKER

sorry...
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of zizi21

ASKER

hi,

thank you..

you are right..it is another problem..i am trying to find out what is it...

once i get to it, i post that part..thanks a milion...thanks