We help IT Professionals succeed at work.

deleting certain values off the end of a word

lazypinkmonkey
on
I am using strtok to extract words from a file. Later I am going to put the words in a table for future use but in the file I'm reading from, some of the words have punctuation at the end. So I only want to add the word to the table, no punctuation. But I keep getting junk when I compile:

Four score and7 years, ago our; fathers? brought forth on
Four
checkTok is Four length is 4
score
checkTok is score length is 5
and7
checkTok is and7 length is 4
years,
checkTok is years, length is 6
years????|||||||||||||||A
ago
checkTok is ago length is 3
our;
checkTok is our; length is 4
our????|1
fathers?
checkTok is fathers? length is 8
fathers????|||||||||||||A
brought
checkTok is brought length is 7
forth
checkTok is forth length is 5
on
checkTok is on length is 2

Here is my program: (is there an easier way to do this?)

int main()
{

ifstream indata;
const MAX = 80;
char bufferarray[MAX];
char *tokenPtr;
int length;
int numofTok = 0;
char *checkTok;
char *newTok;

indata.open("words.txt");
if(indata.fail())
{
     cerr << "Error! File did not open!" << endl;
     exit(1);
}

indata.getline(bufferarray, MAX);
//while(! indata.eof())
//{
cout << bufferarray << endl;

tokenPtr = strtok(bufferarray, " ");

while (tokenPtr != NULL)
{
     cout << tokenPtr << endl;
     length = strlen(tokenPtr);
     checkTok = new char[length + 1];
     strcpy(checkTok, tokenPtr);

    cout << "checkTok is " << checkTok << " length is " << length << endl;
     if(checkTok[length - 1] == '.' || checkTok[length - 1] == ',' || checkTok[length - 1] == '?')
     {
          newTok = new char[length - 1];
          strncpy(newTok, checkTok, length - 1);
          cout << newTok << endl;
     }
     
     if(checkTok[length - 1] == '!' || checkTok[length - 1] == ':' || checkTok[length - 1] == ';')
     {
             newTok = new char[length - 1];
          strncpy(newTok, checkTok, length - 1);
          cout << newTok << endl;
     }

     tokenPtr = strtok(NULL, " ");
}


indata.close();

return 0;
}




Comment
Watch Question

Simply add the punctuation to your list of token separators

char * pstrSeps="\t\n !.,;:";

Then call
   strtok(bufferarray, pstrSeps);

Author

Commented:
I tried the above comment and it doesn't work. If I do the above comment, then I no longer need to get strlen, dynamically allocate, etc. So all I have is:

while (tokenPtr != NULL)
{
    cout << tokenPtr << endl;
   
tokenPtr = strtok(NULL, " ");
}


and the output is:

Four score and7 years, ago our; fathers? brought forth on
Four
score
and7
years,
ago
our;
fathers?
brought
forth
on

Any more suggestions?
Did you put in the comma and the semi-colon and question mark in the separators?

Author

Commented:
Sorry, Sorry, Sorry. The above works, I just did:

strtok(NULL, ",, ., ?, !, :, ;, ,\n, \t");



I got a B for that?

Anyway... you only need the comma in the string once.

Explore More ContentExplore courses, solutions, and other research materials related to this topic.