Seg fault when i try to add a third word to a vector of words

Hi Experts:

So, i'm trying to write a function which receives a string and parses out the words, which are seperated by delimiters.   It's basically a strtok() function.   However, this confounded thing keeps crashing on the third word that I try to add to the word vector.   I left all my bug-hunting statements in the code, just so that anyone who needs them doesn't have to re-type them.   I took most of my comments out as i would suspect them to be highly useless to anyone else; they are a rather incoherent jumble of what I was thinking before and during the actual coding, and bear little relevance to the code itself.   Any help that can be offered on this code would be mighty appreciated.  

Thanks much in advance for any and all comments.

Pierre

==================================
#include <vector>
#include <iostream>
#include <string>

using namespace std;

void tokenize(vector<char*> words, int& numWords, char *line, char *delimiters)
{
      char *startCurWord, *endCurWord;
      char* curDelim;
      int lineLength;
      int delimLength;
   
      curDelim = delimiters;
      startCurWord = line;
   
      //get line length
      lineLength = strlen(line);
   
      //set startCurWord to line
      startCurWord = line;
   
      //loop:
      while((startCurWord - line) < lineLength)
      {
            endCurWord = strpbrk(startCurWord, delimiters);
            if(endCurWord)
            {
                  *endCurWord = '\0';
                  if(endCurWord - startCurWord > 0)
                  {
                        cout << "numWords: " << numWords << endl;
                        cout << "words.size(): " << words.size() << endl;
                        cout << "strlen(startCurWord): " << strlen(startCurWord) << endl;
                        words.push_back(new char[strlen(startCurWord)]);
//                        words.push_back((char*)malloc(strlen(startCurWord) * sizeof(char)));
                        numWords++;
                        words[numWords][0] = '\0';
                        cout << "numWords: " << numWords << endl;
                        cout << "words.size(): " << words.size() << endl;
                        cout << "startCurWord: " << startCurWord << endl;
                        cout << "before: \"" << words[numWords] << "\"" << endl;
                        strcpy(words[numWords], startCurWord);
                        cout << "after: \"" << words[numWords] << "\"" << endl;
                        cout << "numWords: " << numWords << endl;
                        cout << "words.size(): " << words.size() << endl;
                        cout << "===================================" << endl << endl;
                  }
            }
            else
            {
                  cout << "numWords: " << numWords << endl;
                  cout << "words.size(): " << words.size() << endl;
                  cout << "strlen(startCurWord): " << strlen(startCurWord) << endl;
                  words.push_back(new char[strlen(startCurWord)]);
//                  words.push_back((char*)malloc(strlen(startCurWord) * sizeof(char)));
                  numWords++;
                  words[numWords][0] = '\0';
                  cout << "numWords: " << numWords << endl;
                  cout << "words.size(): " << words.size() << endl;
                  cout << "startCurWord: " << startCurWord << endl;
                  cout << "before: \"" << words[numWords] << "\"" << endl;
                  strcpy(words[numWords], startCurWord);
                  cout << "after: \"" << words[numWords] << "\"" << endl;
                  cout << "numWords: " << numWords << endl;
                  cout << "words.size(): " << words.size() << endl;
                  cout << "===================================" << endl << endl;
            }    

            //  set startCurWord to end of current word + 1
            if(endCurWord)
                  startCurWord = endCurWord + 1;
            else
                  startCurWord = line + lineLength;

            //  if startCurWord - line < lineLegth, loop

      }

      return;
}

int main()
{
      int numWords, size;
      int i = 0;
      vector<char*> words;

      numWords = 0;
      char line[] = "aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb ccc ddd";
//      char line[] = "aaa bbb ccc ddd";
//      char line[] = "This ain't a test line.   Pay no attention.";
//      char *line = "This is a test line.   Pay no attention.";
         
      tokenize(words, numWords, line, " ,.");
   
//      cout << numWords << endl;
   
//      for(words.begin(); i < numWords; i++)
//      cerr << words[i] << endl;
   
      system("PAUSE");
   
      return 0;
};    
Diceman_01Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jaime OlivaresSoftware ArchitectCommented:
I suspect on this line:
 words.push_back(new char[strlen(startCurWord)]);

I think must be:
 words.push_back(new char[strlen(startCurWord)+1]);   // reserve an extra space for ending null character

Vector of pointers could be some confusing because you have to pay attention to allocation and deallocation, if you are working with STL vectors, I suggest you to work with std::string objects, don't have to deal with allocation. Something like:

vector<string> words

Good luck,
Jaime.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Diceman_01Author Commented:
Hey Jaime;

thanks very much for your help.   After I asked the question, I kept searching the EE DB and came up wioth this code:

void tokenize(vector<string> &words, int& numWords, char *line, char *delimiters)
{
    char * pToken = strtok(line, delimiters);
    while( pToken )
    {
          words.push_back(pToken);
         pToken = strtok(0, delimiters);
    };     // while  

    numWords = words.size();
}

Well, it does exactly what I need it to do.   I have no idea why I couldn't see the solution earlier.   I suppose this is the reason why one needs to take a break when coding.   At any rate, just out of curiousity, I tried adding the extra space for the null, but to no avail.   However, you're right about the second idea of vector<string>.   Points to you dude.   Thanks again.

Pierre      
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Programming

From novice to tech pro — start learning today.