Solved

problem with using strtok()

Posted on 2004-09-17
8
517 Views
Last Modified: 2008-03-04
strtok() treats ,, (2 commas) as , (1 comma), how can i display empty character between ,,  ? for example

I have a string like " abc,def,ghi,,jkl" (5 columns), the output format should be abc def ghi   jkl ( 5 columns)
but strtok() displays abc def ghi jkl (only 4 columns)

Whats the fix for this type of problem ?
0
Comment
Question by:justinY
8 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 12088864
Just check the lenght of the token that is returned - if it is 0, you have to consecutive delimiters and need to handle that properly.
0
 
LVL 19

Expert Comment

by:drichards
ID: 12089032
No, strtok skips leading delimiters, so if you have a bunch of delimiters in a row, they are all skipped - there is no such thing as a zero-length token.

From  Linux Programmer's Manual:

"The strsep() function was introduced as a replacement for strtok(), since the latter cannot handle empty fields. However, strtok() conforms to ANSI-C and hence is more portable."

You will need to implement your own strtok function or use strsep if your platform supports it.  Really cheap version of custom strtok is given below.  

char *MyStrTok(char *text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
}
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
ID: 12090823
Or that:



#include <iostream>
#include <string>
#include <vector>

using namespace std;

int stringToArray(const string& str, char delim, vector<string>& strArr)
{
    size_t next;
    size_t last = 0;
    while ((next = str.find(delim, last)) != string::npos)
    {
        strArr.push_back(str.substr(last, next - last));
        last = next + 1;
    }
    strArr.push_back(str.substr(last));
    return strArr.size();
}


int main(int argc, char* argv[])
{
     string str = ",abc,def,,ghi,jki,";
     vector<string> strArr;
     int nTok = stringToArray(str, ',', strArr);
     cout << str << " # of tokens " << nTok << endl;
     
     for (int i = 0; i < nTok; i++)
         cout << strArr[i] << endl;

     getline(cin, str);

     return 1;
}

Regards, Alex
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 

Author Comment

by:justinY
ID: 12092604
Thank you very much to all
I have Microsoft VC++.Net standard version. It doesnt support strsep(). So I use custom version strtok() that is MyStrTok() given by drichards. But I am having compiling error. I tried many times, just cannot solve the problem. Can someone take a look at it ? Thank you
Here is my code.

#include <iostream>
#include <fstream>
#include <iomanip>
#include <string>

using namespace std;

const int MaxSize=1024;

//////////////////////////////////
//
// struct for reference data
//
//////////////////////////////////
typedef struct
{
 char keyData[10];
 char valueData[20];
}KeyRef;


//////////////////////////////////////////
// function to lookup value by key
//////////////////////////////////////////
char* findKeyValue(KeyRef* ref, char* k){
      
      for (int i=0;i<MaxSize;i++){
            if (!strcmp(ref[i].keyData,k))
                  return ref[i].valueData;
      }
      return "None";
};


///////////////////////////////
//
// Custom version of strtok()
//
///////////////////////////////

char* MyStrTok(char* text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
};

///////////////////////////////
//
// main
//
///////////////////////////////
main(int arc, char *arv[])
{

ifstream fin("fin.txt");
ofstream fout("fout.txt");
char row_read[512];
char row_elem[10][50];

ifstream fin2("frefer.txt");
ofstream fout2("refer_out.txt");
KeyRef refer_data[MaxSize];
char refer_read[MaxSize];
char refer_row_elem[2][MaxSize];
int i=0,j=0,k=0;

//////////////////////////////////////////
// read refer file into struct of array
//////////////////////////////////////////
while( fin2.getline( refer_read,MaxSize ))
{
      char *tok = strtok(refer_read,",");
      while( tok != NULL) {
            strcpy(refer_row_elem[j],tok);
            tok= strtok(NULL,",");
            j++;
      }
      strcpy(refer_data[k].keyData, refer_row_elem[0]);
      strcpy(refer_data[k].valueData, refer_row_elem[1]);
      j=0;
      k++;
}

/////////////////////////////////////////////////
// output refer file , review data only
/////////////////////////////////////////////////
//for (int q=0;q<k;q++){
//       fout2 << setw(10) << refer_data[q].keyData << setw(10) << refer_data[q].valueData << endl;
//}


///////////////////////////////////////////////////////////////////
// for each line of file  
// assume there are 5 collums in each row in file fin.txt
///////////////////////////////////////////////////////////////////
while( fin.getline( row_read, sizeof( row_read ) ) )
{
      //////////////////////////////////////////
      // parse string with ","
      //////////////////////////////////////////
      char *token = MyStrTok( row_read, "," );
      //char *token = strsep( row_read, "," );
      while( token != NULL )
      {
        /* While there are tokens in "string" */
        strcpy(row_elem[i], token );
        /* Get next token: */
        token = strtok( NULL, "," );
        i++;
      }

      //////////////////////////////////////////////////////////////////////
      // get reference value by looking at key in reference file
      //////////////////////////////////////////////////////////////////////
      char* valueFound;
      valueFound = findKeyValue(refer_data, row_elem[0]);
      cout << " value found - " << valueFound << endl;

      //////////////////////////////////////////////////////////////////////
      // output to file, first colmn replaced with value from reference file
      //////////////////////////////////////////////////////////////////////
      fout<<setw(10)      << valueFound
          <<setw(10)  << row_elem[4]
          <<setw(10)      << row_elem[3]
          <<setw(10)      << row_elem[2]
          <<setw(10)      << row_elem[1]
          <<endl;

      i=0;
}

return 0;
}
0
 
LVL 19

Expert Comment

by:drichards
ID: 12092672
1) Make sure you turn off precompiled headers for this file or add '#include "stdafx.h"' at the top of the file.

2) '     char *token = MyStrTok( row_read, "," );' should be:  char *token = MyStrTok( row_read, ',' );
Note the single quotes - my version of strtok takes a single character as a delimiter.  The code is a lot simpler if you don't need multiple delimiters.

If you need to turn off precompiled headers, right click on the source file in the Solution Explorer and select "Properties". Go to the C/C++->Precompiled Headers page and choose "Not Using Precompiled Headers" on the "Create/Use Precompiled Header" line.
0
 

Author Comment

by:justinY
ID: 12092893
drichards,
single quotes works, but other problems.
1. #include "stdafx.h" cannot be found in my vc7\include folder.
2. precompiled headers have been turn off.
3. without '#include "stdafx.h"', I can compile my code, but the output is funny. It only catch one column.
I post my fin, and frefer files. you run the code and will see the output.

fin:
id,notes1,notes2,notes3,account name,amount
111,aaa,a3,a4,,100000
222,bbb,b3,b4,,200000
333,ccc,c3,c4,cccc5,300000
444,ddd,d3,d4,dddd5,400000
555,eee,e3,e4,eeee5,500000
666,fff,f3,f4,ffff5,600000
777,ggg,g3,g4,gggg5,700000
888,hhh,h3,h4,hhhh5,800000
999,iii,i3,i4,iiii5,900000


frefer:
111,111aaa
222,222bbb
333,333ccc
444,444ddd
555,555eee
666,666fff
777,777ggg
888,888hhh
999,999iii


Thanks and waiting online
0
 
LVL 19

Accepted Solution

by:
drichards earned 500 total points
ID: 12093371
While parsing fin, you use MyStrTok to start but then use strtok in the loop.  Change:

       token = strtok( NULL, "," );
to
       token = MyStrTok( NULL, ',' );

in the 'while (fin.getline(...))' loop.

There is also a minor logic error in that you try to look up the header line info and get a result of 'None' which is prined out as the header if the ID column.  You'll want to fix that.  Probably the first time through (this will be the header line) don't do 'findKeyValue'.
0
 
LVL 55

Expert Comment

by:Jaime Olivares
ID: 12096767
Here is an implementation of strsep() function:
http://www.greatsnakes.com/Sepal/d4/d0/strsep_8c-source.html
0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
C++ - Loading Managed Assembly From Memory in Unmanaged Process 25 473
ADO Memory leak with DELPHI 2007 37 181
FMX enumerated colours 2 99
sorting efficency of sorting algorithm 30 113
Errors will happen. It is a fact of life for the programmer. How and when errors are detected have a great impact on quality and cost of a product. It is better to detect errors at compile time, when possible and practical. Errors that make their wa…
This article shows you how to optimize memory allocations in C++ using placement new. Applicable especially to usecases dealing with creation of large number of objects. A brief on problem: Lets take example problem for simplicity: - I have a G…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will be introduced to the member functions push_back and pop_back of the vector class. The video will teach the difference between the two as well as how to use each one along with its functionality.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question