Solved

problem with using strtok()

Posted on 2004-09-17
8
543 Views
Last Modified: 2008-03-04
strtok() treats ,, (2 commas) as , (1 comma), how can i display empty character between ,,  ? for example

I have a string like " abc,def,ghi,,jkl" (5 columns), the output format should be abc def ghi   jkl ( 5 columns)
but strtok() displays abc def ghi jkl (only 4 columns)

Whats the fix for this type of problem ?
0
Comment
Question by:justinY
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 12088864
Just check the lenght of the token that is returned - if it is 0, you have to consecutive delimiters and need to handle that properly.
0
 
LVL 19

Expert Comment

by:drichards
ID: 12089032
No, strtok skips leading delimiters, so if you have a bunch of delimiters in a row, they are all skipped - there is no such thing as a zero-length token.

From  Linux Programmer's Manual:

"The strsep() function was introduced as a replacement for strtok(), since the latter cannot handle empty fields. However, strtok() conforms to ANSI-C and hence is more portable."

You will need to implement your own strtok function or use strsep if your platform supports it.  Really cheap version of custom strtok is given below.  

char *MyStrTok(char *text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
}
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
ID: 12090823
Or that:



#include <iostream>
#include <string>
#include <vector>

using namespace std;

int stringToArray(const string& str, char delim, vector<string>& strArr)
{
    size_t next;
    size_t last = 0;
    while ((next = str.find(delim, last)) != string::npos)
    {
        strArr.push_back(str.substr(last, next - last));
        last = next + 1;
    }
    strArr.push_back(str.substr(last));
    return strArr.size();
}


int main(int argc, char* argv[])
{
     string str = ",abc,def,,ghi,jki,";
     vector<string> strArr;
     int nTok = stringToArray(str, ',', strArr);
     cout << str << " # of tokens " << nTok << endl;
     
     for (int i = 0; i < nTok; i++)
         cout << strArr[i] << endl;

     getline(cin, str);

     return 1;
}

Regards, Alex
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:justinY
ID: 12092604
Thank you very much to all
I have Microsoft VC++.Net standard version. It doesnt support strsep(). So I use custom version strtok() that is MyStrTok() given by drichards. But I am having compiling error. I tried many times, just cannot solve the problem. Can someone take a look at it ? Thank you
Here is my code.

#include <iostream>
#include <fstream>
#include <iomanip>
#include <string>

using namespace std;

const int MaxSize=1024;

//////////////////////////////////
//
// struct for reference data
//
//////////////////////////////////
typedef struct
{
 char keyData[10];
 char valueData[20];
}KeyRef;


//////////////////////////////////////////
// function to lookup value by key
//////////////////////////////////////////
char* findKeyValue(KeyRef* ref, char* k){
      
      for (int i=0;i<MaxSize;i++){
            if (!strcmp(ref[i].keyData,k))
                  return ref[i].valueData;
      }
      return "None";
};


///////////////////////////////
//
// Custom version of strtok()
//
///////////////////////////////

char* MyStrTok(char* text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
};

///////////////////////////////
//
// main
//
///////////////////////////////
main(int arc, char *arv[])
{

ifstream fin("fin.txt");
ofstream fout("fout.txt");
char row_read[512];
char row_elem[10][50];

ifstream fin2("frefer.txt");
ofstream fout2("refer_out.txt");
KeyRef refer_data[MaxSize];
char refer_read[MaxSize];
char refer_row_elem[2][MaxSize];
int i=0,j=0,k=0;

//////////////////////////////////////////
// read refer file into struct of array
//////////////////////////////////////////
while( fin2.getline( refer_read,MaxSize ))
{
      char *tok = strtok(refer_read,",");
      while( tok != NULL) {
            strcpy(refer_row_elem[j],tok);
            tok= strtok(NULL,",");
            j++;
      }
      strcpy(refer_data[k].keyData, refer_row_elem[0]);
      strcpy(refer_data[k].valueData, refer_row_elem[1]);
      j=0;
      k++;
}

/////////////////////////////////////////////////
// output refer file , review data only
/////////////////////////////////////////////////
//for (int q=0;q<k;q++){
//       fout2 << setw(10) << refer_data[q].keyData << setw(10) << refer_data[q].valueData << endl;
//}


///////////////////////////////////////////////////////////////////
// for each line of file  
// assume there are 5 collums in each row in file fin.txt
///////////////////////////////////////////////////////////////////
while( fin.getline( row_read, sizeof( row_read ) ) )
{
      //////////////////////////////////////////
      // parse string with ","
      //////////////////////////////////////////
      char *token = MyStrTok( row_read, "," );
      //char *token = strsep( row_read, "," );
      while( token != NULL )
      {
        /* While there are tokens in "string" */
        strcpy(row_elem[i], token );
        /* Get next token: */
        token = strtok( NULL, "," );
        i++;
      }

      //////////////////////////////////////////////////////////////////////
      // get reference value by looking at key in reference file
      //////////////////////////////////////////////////////////////////////
      char* valueFound;
      valueFound = findKeyValue(refer_data, row_elem[0]);
      cout << " value found - " << valueFound << endl;

      //////////////////////////////////////////////////////////////////////
      // output to file, first colmn replaced with value from reference file
      //////////////////////////////////////////////////////////////////////
      fout<<setw(10)      << valueFound
          <<setw(10)  << row_elem[4]
          <<setw(10)      << row_elem[3]
          <<setw(10)      << row_elem[2]
          <<setw(10)      << row_elem[1]
          <<endl;

      i=0;
}

return 0;
}
0
 
LVL 19

Expert Comment

by:drichards
ID: 12092672
1) Make sure you turn off precompiled headers for this file or add '#include "stdafx.h"' at the top of the file.

2) '     char *token = MyStrTok( row_read, "," );' should be:  char *token = MyStrTok( row_read, ',' );
Note the single quotes - my version of strtok takes a single character as a delimiter.  The code is a lot simpler if you don't need multiple delimiters.

If you need to turn off precompiled headers, right click on the source file in the Solution Explorer and select "Properties". Go to the C/C++->Precompiled Headers page and choose "Not Using Precompiled Headers" on the "Create/Use Precompiled Header" line.
0
 

Author Comment

by:justinY
ID: 12092893
drichards,
single quotes works, but other problems.
1. #include "stdafx.h" cannot be found in my vc7\include folder.
2. precompiled headers have been turn off.
3. without '#include "stdafx.h"', I can compile my code, but the output is funny. It only catch one column.
I post my fin, and frefer files. you run the code and will see the output.

fin:
id,notes1,notes2,notes3,account name,amount
111,aaa,a3,a4,,100000
222,bbb,b3,b4,,200000
333,ccc,c3,c4,cccc5,300000
444,ddd,d3,d4,dddd5,400000
555,eee,e3,e4,eeee5,500000
666,fff,f3,f4,ffff5,600000
777,ggg,g3,g4,gggg5,700000
888,hhh,h3,h4,hhhh5,800000
999,iii,i3,i4,iiii5,900000


frefer:
111,111aaa
222,222bbb
333,333ccc
444,444ddd
555,555eee
666,666fff
777,777ggg
888,888hhh
999,999iii


Thanks and waiting online
0
 
LVL 19

Accepted Solution

by:
drichards earned 500 total points
ID: 12093371
While parsing fin, you use MyStrTok to start but then use strtok in the loop.  Change:

       token = strtok( NULL, "," );
to
       token = MyStrTok( NULL, ',' );

in the 'while (fin.getline(...))' loop.

There is also a minor logic error in that you try to look up the header line info and get a result of 'None' which is prined out as the header if the ID column.  You'll want to fix that.  Probably the first time through (this will be the header line) don't do 'findKeyValue'.
0
 
LVL 55

Expert Comment

by:Jaime Olivares
ID: 12096767
Here is an implementation of strsep() function:
http://www.greatsnakes.com/Sepal/d4/d0/strsep_8c-source.html
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Fast access to array, using indexes, smart iterative search 2 132
Problem with SqlConnection 4 193
What is sub-make ? 2 103
VS2015 Redefinition errors 4 103
This article will show you some of the more useful Standard Template Library (STL) algorithms through the use of working examples.  You will learn about how these algorithms fit into the STL architecture, how they work with STL containers, and why t…
Introduction This article is a continuation of the C/C++ Visual Studio Express debugger series. Part 1 provided a quick start guide in using the debugger. Part 2 focused on additional topics in breakpoints. As your assignments become a little more …
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question