Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

problem with using strtok()

Posted on 2004-09-17
8
Medium Priority
?
565 Views
Last Modified: 2008-03-04
strtok() treats ,, (2 commas) as , (1 comma), how can i display empty character between ,,  ? for example

I have a string like " abc,def,ghi,,jkl" (5 columns), the output format should be abc def ghi   jkl ( 5 columns)
but strtok() displays abc def ghi jkl (only 4 columns)

Whats the fix for this type of problem ?
0
Comment
Question by:justinY
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 12088864
Just check the lenght of the token that is returned - if it is 0, you have to consecutive delimiters and need to handle that properly.
0
 
LVL 19

Expert Comment

by:drichards
ID: 12089032
No, strtok skips leading delimiters, so if you have a bunch of delimiters in a row, they are all skipped - there is no such thing as a zero-length token.

From  Linux Programmer's Manual:

"The strsep() function was introduced as a replacement for strtok(), since the latter cannot handle empty fields. However, strtok() conforms to ANSI-C and hence is more portable."

You will need to implement your own strtok function or use strsep if your platform supports it.  Really cheap version of custom strtok is given below.  

char *MyStrTok(char *text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
}
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
ID: 12090823
Or that:



#include <iostream>
#include <string>
#include <vector>

using namespace std;

int stringToArray(const string& str, char delim, vector<string>& strArr)
{
    size_t next;
    size_t last = 0;
    while ((next = str.find(delim, last)) != string::npos)
    {
        strArr.push_back(str.substr(last, next - last));
        last = next + 1;
    }
    strArr.push_back(str.substr(last));
    return strArr.size();
}


int main(int argc, char* argv[])
{
     string str = ",abc,def,,ghi,jki,";
     vector<string> strArr;
     int nTok = stringToArray(str, ',', strArr);
     cout << str << " # of tokens " << nTok << endl;
     
     for (int i = 0; i < nTok; i++)
         cout << strArr[i] << endl;

     getline(cin, str);

     return 1;
}

Regards, Alex
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:justinY
ID: 12092604
Thank you very much to all
I have Microsoft VC++.Net standard version. It doesnt support strsep(). So I use custom version strtok() that is MyStrTok() given by drichards. But I am having compiling error. I tried many times, just cannot solve the problem. Can someone take a look at it ? Thank you
Here is my code.

#include <iostream>
#include <fstream>
#include <iomanip>
#include <string>

using namespace std;

const int MaxSize=1024;

//////////////////////////////////
//
// struct for reference data
//
//////////////////////////////////
typedef struct
{
 char keyData[10];
 char valueData[20];
}KeyRef;


//////////////////////////////////////////
// function to lookup value by key
//////////////////////////////////////////
char* findKeyValue(KeyRef* ref, char* k){
      
      for (int i=0;i<MaxSize;i++){
            if (!strcmp(ref[i].keyData,k))
                  return ref[i].valueData;
      }
      return "None";
};


///////////////////////////////
//
// Custom version of strtok()
//
///////////////////////////////

char* MyStrTok(char* text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
};

///////////////////////////////
//
// main
//
///////////////////////////////
main(int arc, char *arv[])
{

ifstream fin("fin.txt");
ofstream fout("fout.txt");
char row_read[512];
char row_elem[10][50];

ifstream fin2("frefer.txt");
ofstream fout2("refer_out.txt");
KeyRef refer_data[MaxSize];
char refer_read[MaxSize];
char refer_row_elem[2][MaxSize];
int i=0,j=0,k=0;

//////////////////////////////////////////
// read refer file into struct of array
//////////////////////////////////////////
while( fin2.getline( refer_read,MaxSize ))
{
      char *tok = strtok(refer_read,",");
      while( tok != NULL) {
            strcpy(refer_row_elem[j],tok);
            tok= strtok(NULL,",");
            j++;
      }
      strcpy(refer_data[k].keyData, refer_row_elem[0]);
      strcpy(refer_data[k].valueData, refer_row_elem[1]);
      j=0;
      k++;
}

/////////////////////////////////////////////////
// output refer file , review data only
/////////////////////////////////////////////////
//for (int q=0;q<k;q++){
//       fout2 << setw(10) << refer_data[q].keyData << setw(10) << refer_data[q].valueData << endl;
//}


///////////////////////////////////////////////////////////////////
// for each line of file  
// assume there are 5 collums in each row in file fin.txt
///////////////////////////////////////////////////////////////////
while( fin.getline( row_read, sizeof( row_read ) ) )
{
      //////////////////////////////////////////
      // parse string with ","
      //////////////////////////////////////////
      char *token = MyStrTok( row_read, "," );
      //char *token = strsep( row_read, "," );
      while( token != NULL )
      {
        /* While there are tokens in "string" */
        strcpy(row_elem[i], token );
        /* Get next token: */
        token = strtok( NULL, "," );
        i++;
      }

      //////////////////////////////////////////////////////////////////////
      // get reference value by looking at key in reference file
      //////////////////////////////////////////////////////////////////////
      char* valueFound;
      valueFound = findKeyValue(refer_data, row_elem[0]);
      cout << " value found - " << valueFound << endl;

      //////////////////////////////////////////////////////////////////////
      // output to file, first colmn replaced with value from reference file
      //////////////////////////////////////////////////////////////////////
      fout<<setw(10)      << valueFound
          <<setw(10)  << row_elem[4]
          <<setw(10)      << row_elem[3]
          <<setw(10)      << row_elem[2]
          <<setw(10)      << row_elem[1]
          <<endl;

      i=0;
}

return 0;
}
0
 
LVL 19

Expert Comment

by:drichards
ID: 12092672
1) Make sure you turn off precompiled headers for this file or add '#include "stdafx.h"' at the top of the file.

2) '     char *token = MyStrTok( row_read, "," );' should be:  char *token = MyStrTok( row_read, ',' );
Note the single quotes - my version of strtok takes a single character as a delimiter.  The code is a lot simpler if you don't need multiple delimiters.

If you need to turn off precompiled headers, right click on the source file in the Solution Explorer and select "Properties". Go to the C/C++->Precompiled Headers page and choose "Not Using Precompiled Headers" on the "Create/Use Precompiled Header" line.
0
 

Author Comment

by:justinY
ID: 12092893
drichards,
single quotes works, but other problems.
1. #include "stdafx.h" cannot be found in my vc7\include folder.
2. precompiled headers have been turn off.
3. without '#include "stdafx.h"', I can compile my code, but the output is funny. It only catch one column.
I post my fin, and frefer files. you run the code and will see the output.

fin:
id,notes1,notes2,notes3,account name,amount
111,aaa,a3,a4,,100000
222,bbb,b3,b4,,200000
333,ccc,c3,c4,cccc5,300000
444,ddd,d3,d4,dddd5,400000
555,eee,e3,e4,eeee5,500000
666,fff,f3,f4,ffff5,600000
777,ggg,g3,g4,gggg5,700000
888,hhh,h3,h4,hhhh5,800000
999,iii,i3,i4,iiii5,900000


frefer:
111,111aaa
222,222bbb
333,333ccc
444,444ddd
555,555eee
666,666fff
777,777ggg
888,888hhh
999,999iii


Thanks and waiting online
0
 
LVL 19

Accepted Solution

by:
drichards earned 2000 total points
ID: 12093371
While parsing fin, you use MyStrTok to start but then use strtok in the loop.  Change:

       token = strtok( NULL, "," );
to
       token = MyStrTok( NULL, ',' );

in the 'while (fin.getline(...))' loop.

There is also a minor logic error in that you try to look up the header line info and get a result of 'None' which is prined out as the header if the ID column.  You'll want to fix that.  Probably the first time through (this will be the header line) don't do 'findKeyValue'.
0
 
LVL 55

Expert Comment

by:Jaime Olivares
ID: 12096767
Here is an implementation of strsep() function:
http://www.greatsnakes.com/Sepal/d4/d0/strsep_8c-source.html
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When writing generic code, using template meta-programming techniques, it is sometimes useful to know if a type is convertible to another type. A good example of when this might be is if you are writing diagnostic instrumentation for code to generat…
IntroductionThis article is the second in a three part article series on the Visual Studio 2008 Debugger.  It provides tips in setting and using breakpoints. If not familiar with this debugger, you can find a basic introduction in the EE article loc…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
The viewer will be introduced to the member functions push_back and pop_back of the vector class. The video will teach the difference between the two as well as how to use each one along with its functionality.

721 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question