Tokenizer in C?

Posted on 2004-11-18
Last Modified: 2010-04-15
below is my code, I try to copy a string into another string when i get first index position of the string and the last index of the string when it gets to the comman.  Unfortunately,  i do not get that idea
please help me out. I am new to C.

#define SIZE 81

void getInput(char line[]);
int isDelimiter(char c[]);

int maxPos;
char inputString[SIZE];
int newPos;

int main(void)
{      int i;
      char tempLastName[20];

      printf("Please enter your full name folllows this format: \n");
      printf("Last name, first name, V >> ");

    maxPos = strlen(inputString);
    newPos = isDelimiter(inputString);
      printf("newPos %d\n\n", newPos);

      for(i = 0; i < newPos; i++)
            //printf("%c", inputString[i]);
            if(i < newPos)
                  tempLastName[i] = inputString[i];

      printf("\n\nLast Name %s \n\n", tempLastName);
//      printf("%s\n\n", inputString);
//      printf("Max number of character inside the string is: %d\n\n", maxPos);

      return 0;


void getInput(char line[])
      int i = 0;
      while(i < 81 && (line[i++] = getchar()) != '\n')

      line[i] = '\0';



int isDelimiter(char c[])
      int count = 0;
      int i;
      for(i = 0; c[i] != ','; i++)

      return count;

Question by:thayboi3000

    Expert Comment

    Everything looks fine, except for two things:

    1.    tempLastName[newPos];   -- this line doesn't do anything.  Not sure what you were looking to do here.
    2.   you forgot to put the zero terminator on the tempLastName after your "for" loop.

    Change it to this:

         for(i = 0; i < newPos; i++)
              //printf("%c", inputString[i]);
              if(i < newPos)
                   tempLastName[i] = inputString[i];
         tempLastName [i] = '\0';

    and you'll be fine.

    LVL 45

    Accepted Solution

    Use strtok function

    An example of a strtok use

    Also refer to the man page

    Your code will go something like

    char string[128] = "blah blah";
    char delim[] = ","; <-- put all your delimiter characters here
    int i = 1;
    char * temp = strtok ( string, delim );

    while ( temp )
             printf ( " token %d is %s", i, temp );
             temp = strtok (NULL, delim);
    LVL 16

    Assisted Solution

    Actually, you dont need the 'if' in the loop either:

         for(i = 0; i < newPos; i++)
            tempLastName[i] = inputString[i];
         tempLastName [i] = '\0';

    which can be further improved to:

       strncpy ( tempLastName, inputString, newPos );

    LVL 22

    Assisted Solution

    ... you probably should enhance the isDelimiter function to check for end of string, just in case there's no comma.
    Otherwise it will scan past the end of the string and you'll get a lot of garbage or a KABOOM!


    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    6 Surprising Benefits of Threat Intelligence

    All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

    Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode ( They will have you believe that Unicode requires you to use…
    This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
    The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.
    The goal of this video is to provide viewers with basic examples to understand and use switch statements in the C programming language.

    779 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    16 Experts available now in Live!

    Get 1:1 Help Now