Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


Strtok function

Posted on 2003-11-19
Medium Priority
Last Modified: 2012-06-27
If anyone writes this before Saturday and it passes my main.c function test (can be requested) it’s worth 500 points.

The motivation behind this program is to implement the functionality of the tokenizer that is a part of all compilers, interpreters, parsers, and scripting languages. For example, the C parser identifies the following comma-separated list of tokens (which are the smallest elements of a C program that are meaningful to the compiler) from the expression a=i+++j: a, =, i, ++, +, j. For our program, we will use the tokenizer to interpret the contents of a null-terminated string as words by filtering out the non-alphabetical characters. The following is a description of the function that you must implement:
Write a function StringToken() with the following prototype:
char* StringToken (char *str, const char *set);
StringToken mimics the behavior of function strtok declared in the C standard library file string.h. The explanation for StringToken is a function that searches str for tokens delimited by characters from set. A sequence of calls of StringToken(str, set) splits str into tokens, each delimited by a character from set. The first call in a sequence has a non-null str. It finds the first token in str consisting of characters not in set; it terminates that by overwriting the next character of str with ‘\0’ and returns a pointer to the token. Each subsequent call, indicates by a null value of str, returns the next such token, searching from just past the end of the previous one. StringToken returns null when no further token is found. The string set may be different on each call.
Sample Program:
Here is a sample program that illustrates the use of StringToken to determine the number of words in a null-terminated string by filtering out certain non-alphabetical characters:
#include <stdio.h>
#define LINE_LEN            1024
char * StringToken(char *str, const char *set)
            /* … */
int main(void)
      Note the use of the escape sequence \" to represent double quotation marks within
the string. An escape sequence begins with the backslash '\' and is used to represent certain nonprintable characters ( for example: '\0', '\a' (bell)),
the single and double quotation marks and the backslash characters ('\'', '\"', '\\').
            char line[LINE_LEN] = “\”My goodness,\” she said, \”Is that right?\””;
            /* Set of non-alphabetic characters */
char set[] = “ .,?!’\”\t\n”;
            char *word = StringToken(line, set); /* find first word */
            while (word)
                  printf("<%s>\n", word);
                  word = StringToken(0, set); /* find next word */
            return 0;
The results of the above program are the words (delimited by <>):
The function StringToken requires a variable defined with the storage-class specifier static. Block-scope variables declared with the static storage class specifier have static extent (as opposed to local extent, signified by auto). An object is said to have static extent when it is allocated storage at or before the beginning of program execution and the storage remains allocated until program termination. Consider the following example:
#include <stdio.h>
      void foo(int *p)
            static int k = 5;
            *p += k;
      int main(void)
            int k = 0, *p = &k;
            printf(“k = %d\n”, k);
            printf(“k = %d\n”, k);
            printf(“k = %d\n”, k);
The three values displayed by the program would be 6, 13, and 21, respectively. Note that object k with block-scope in function foo has static extent and is initialized with the integer value 5 at or before the beginning of program execution
Your submission would consist of two files: a C source file and a header file.
Your submission must not use any global variables.
 No Standard C Library functions are to be used in your submission
Ensure that there are no statements that perform I/O.
Question by:postal21
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
LVL 45

Expert Comment

ID: 9777380
sorry postal21 ... EE membership agreement forbids us from doing your homework ...
You have to do it on your own ...
If you are stuck with a specific problem, we can give you a push but we cannot do it for you
LVL 45

Expert Comment

ID: 9777412
that been said, perhaps you may find strcspn() function useful

man strcspn for more details
LVL 22

Accepted Solution

grg99 earned 2000 total points
ID: 9782242
We can't do the whole thing, but here's a  outline.
Note , several boundary conditions and off-by-ones left in as
an exercise. :)

char * p = NULL;

char* StringToken (char *str, const char *set)
if( str != NULL ) p = str;

Start = p;
while( Isin( *p++, set ) ) p++;
*p = '\0';
return( Start );

int Isin( char Target, cha * List )
int Found;
Found = 0;
while( ! Found && *List != '\0' ) Found = *List++ == Target
return( Found );


Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
The goal of this video is to provide viewers with basic examples to understand how to use strings and some functions related to them in the C programming language.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question