How does "strtok()" search in a given character string line?

Posted on 2003-03-27
Medium Priority
Last Modified: 2008-02-01
Hello Everyone!  

I have a following up question to the same code which is posted about a week ago.  My ex-neighbor wrote the following C-Program to organize and chop a big ASCII text down into a convenient set of files several years ago.   Now, I want to understand the Code completely by myself.   I mostly understand the vocabularies such as *inf, fopen, feof, and strncmp as well as the "!" symbol.   This time, my question is how to use "strntok()" and how "strntok()" works when it is scanning in a incoming character string.  The code have two occasions.   Could someone explain these codes in comprehensive words?  I also post the part of Data (after the CODE, all way down), so that you may see the code clearer.

              t2=strtok(NULL," ");
Mitsuru (in Los Angeles)

======== THE CODE FOLLOWS =======================
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#define BUF_LEN 1024

void main(int argc, char *argv[])
    FILE *inf,*outf;
    char text[BUF_LEN], tcopy[BUF_LEN], *t2;

    if (argc<2) {
         printf("Need a filename!\n");
    if (!(inf=fopen(argv[1],"rt"))) {
         printf("Could not open input file [%s]\n",argv[1]);
    while (!feof(inf)) {
         if (!strncmp(text,"UI",2)) {
              t2=strtok(NULL," ");
              printf("found UI : %s\n",t2);
              if(!(outf=fopen(t2,"wt"))) {
                   printf("Could not open output file: %s\n",t2);

========A part sample of DATA FILE follows.==========

UI - Abreu-CT

Francis L. Abreu Charitable Trust
c/o SunTrust Banks, Inc.  
P.O. Box 4655, MC 221  
Atlanta, GA  30302-4655
Contact: Danah Craft, Asst. V.P., SunTrust Banks, Inc.  
URL: http://www.abreufoundation.org 
EIN: 586130937


The AEC Trust
50 Hurt Plz., Ste. 900  
Atlanta, GA  30303
Contact: Christine Tryba-Cofrin
FAX: (404) 577-9490
Donor(s):  Members of the Cofrin family.
EIN: 366725987

UI - Alberto

Alberto-Culver Company Contributions Program
2525 Armitage Ave.  
Melrose Park, IL  60160-1163
Telephone: (708) 450-3000
Contact: Nancy Shields, Mgr., Corp. Comm.  
FAX: (708) 450-3354

Question by:mkido
  • 2

Accepted Solution

gj62 earned 200 total points
ID: 8221396
strtok finds tokens that are separated by a specific character, or one of a set of characters (often referred to as delimiters).  

Take for example the following line of first names:

char names[] = "glenn;robert;george;francis";
char *name1,*name2,*name3,*name4;

If you wanted to parse these names into separate strings, you would do:

name1 = strtok(names,";"); /*return the FIRST string up to the first semicolon (delimiter), or end of string.  Do not return the first delimiter*/

name2 = strtok(NULL, ";"); /*parse the rest*/
name3 = strtok(NULL, ";");
name4 = strtok(NULL, ";");

Notice you don't repeat names, rather you use NULL, and strtok "remembers" the string you were working on (there's problems here in threaded apps, but let's not worry about that right now).  Strtok allows you to have more than one delimited, I could do the following:

char names[] = "glenn robert,george;francis";
char *name1,*name2,*name3,*name4;

name1 = strtok(names,"; ,"); /*now we can use a comma, a space, OR a semicolon as a delimiter*/
name2 = strtok(NULL, ";"); /*parse the rest*/
name3 = strtok(NULL, ";");
name4 = strtok(NULL, ";");

You can also change the delimiter on subsequent calls to be whatever you wish, as your program above does.  This can be done successfully only when the file follows a rigid layout structure.

Lastly, strtok works by actually modifying the string it works on.  What it returns is a pointer to the *beginning* of the next token, and replaces the delimiter with NULL.

Originally, my string looked like:


if you said

printf("%s", names), it would print out just like above...

When strtok got through with it, it looked like:


with each 0 being null (which is how C knows it has come to the end of the string).  If you now said:

printf("%s", names), it would print:


that's it - because there is now a null there, and that's where C thinks the string stops.  But name1 points to the "r" in robert, so you could printf name1, and it would print


hope this helps...

Author Comment

ID: 8229344
Thank you, qj62.   I will test a few variations from your explanation, and see how srntok() works.  One moment.

Author Comment

ID: 8239338
Dear gj62:

Your answer is so gorgeous.  I saw everything you talked about.  Thank you.  mkido

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
Examines three attack vectors, specifically, the different types of malware used in malicious attacks, web application attacks, and finally, network based attacks.  Concludes by examining the means of securing and protecting critical systems and inf…
The goal of this video is to provide viewers with basic examples to understand and use pointers in the C programming language.
The goal of this video is to provide viewers with basic examples to understand and use structures in the C programming language.

621 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question