Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

problem with using strtok()

Posted on 2004-09-17
8
527 Views
Last Modified: 2008-03-04
strtok() treats ,, (2 commas) as , (1 comma), how can i display empty character between ,,  ? for example

I have a string like " abc,def,ghi,,jkl" (5 columns), the output format should be abc def ghi   jkl ( 5 columns)
but strtok() displays abc def ghi jkl (only 4 columns)

Whats the fix for this type of problem ?
0
Comment
Question by:justinY
8 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 12088864
Just check the lenght of the token that is returned - if it is 0, you have to consecutive delimiters and need to handle that properly.
0
 
LVL 19

Expert Comment

by:drichards
ID: 12089032
No, strtok skips leading delimiters, so if you have a bunch of delimiters in a row, they are all skipped - there is no such thing as a zero-length token.

From  Linux Programmer's Manual:

"The strsep() function was introduced as a replacement for strtok(), since the latter cannot handle empty fields. However, strtok() conforms to ANSI-C and hence is more portable."

You will need to implement your own strtok function or use strsep if your platform supports it.  Really cheap version of custom strtok is given below.  

char *MyStrTok(char *text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
}
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
ID: 12090823
Or that:



#include <iostream>
#include <string>
#include <vector>

using namespace std;

int stringToArray(const string& str, char delim, vector<string>& strArr)
{
    size_t next;
    size_t last = 0;
    while ((next = str.find(delim, last)) != string::npos)
    {
        strArr.push_back(str.substr(last, next - last));
        last = next + 1;
    }
    strArr.push_back(str.substr(last));
    return strArr.size();
}


int main(int argc, char* argv[])
{
     string str = ",abc,def,,ghi,jki,";
     vector<string> strArr;
     int nTok = stringToArray(str, ',', strArr);
     cout << str << " # of tokens " << nTok << endl;
     
     for (int i = 0; i < nTok; i++)
         cout << strArr[i] << endl;

     getline(cin, str);

     return 1;
}

Regards, Alex
0
Networking for the Cloud Era

Join Microsoft and Riverbed for a discussion and demonstration of enhancements to SteelConnect:
-One-click orchestration and cloud connectivity in Azure environments
-Tight integration of SD-WAN and WAN optimization capabilities
-Scalability and resiliency equal to a data center

 

Author Comment

by:justinY
ID: 12092604
Thank you very much to all
I have Microsoft VC++.Net standard version. It doesnt support strsep(). So I use custom version strtok() that is MyStrTok() given by drichards. But I am having compiling error. I tried many times, just cannot solve the problem. Can someone take a look at it ? Thank you
Here is my code.

#include <iostream>
#include <fstream>
#include <iomanip>
#include <string>

using namespace std;

const int MaxSize=1024;

//////////////////////////////////
//
// struct for reference data
//
//////////////////////////////////
typedef struct
{
 char keyData[10];
 char valueData[20];
}KeyRef;


//////////////////////////////////////////
// function to lookup value by key
//////////////////////////////////////////
char* findKeyValue(KeyRef* ref, char* k){
      
      for (int i=0;i<MaxSize;i++){
            if (!strcmp(ref[i].keyData,k))
                  return ref[i].valueData;
      }
      return "None";
};


///////////////////////////////
//
// Custom version of strtok()
//
///////////////////////////////

char* MyStrTok(char* text, const char del)
{
    static char *curTok = NULL;
    char *result = NULL;
    if ( text != NULL )
    {
        curTok = text;
    }
    result = curTok;
    if ( curTok != NULL )
    {
        curTok = ::strchr(curTok, del);
        if ( curTok != NULL ) *curTok++ = 0;
    }

    return result;
};

///////////////////////////////
//
// main
//
///////////////////////////////
main(int arc, char *arv[])
{

ifstream fin("fin.txt");
ofstream fout("fout.txt");
char row_read[512];
char row_elem[10][50];

ifstream fin2("frefer.txt");
ofstream fout2("refer_out.txt");
KeyRef refer_data[MaxSize];
char refer_read[MaxSize];
char refer_row_elem[2][MaxSize];
int i=0,j=0,k=0;

//////////////////////////////////////////
// read refer file into struct of array
//////////////////////////////////////////
while( fin2.getline( refer_read,MaxSize ))
{
      char *tok = strtok(refer_read,",");
      while( tok != NULL) {
            strcpy(refer_row_elem[j],tok);
            tok= strtok(NULL,",");
            j++;
      }
      strcpy(refer_data[k].keyData, refer_row_elem[0]);
      strcpy(refer_data[k].valueData, refer_row_elem[1]);
      j=0;
      k++;
}

/////////////////////////////////////////////////
// output refer file , review data only
/////////////////////////////////////////////////
//for (int q=0;q<k;q++){
//       fout2 << setw(10) << refer_data[q].keyData << setw(10) << refer_data[q].valueData << endl;
//}


///////////////////////////////////////////////////////////////////
// for each line of file  
// assume there are 5 collums in each row in file fin.txt
///////////////////////////////////////////////////////////////////
while( fin.getline( row_read, sizeof( row_read ) ) )
{
      //////////////////////////////////////////
      // parse string with ","
      //////////////////////////////////////////
      char *token = MyStrTok( row_read, "," );
      //char *token = strsep( row_read, "," );
      while( token != NULL )
      {
        /* While there are tokens in "string" */
        strcpy(row_elem[i], token );
        /* Get next token: */
        token = strtok( NULL, "," );
        i++;
      }

      //////////////////////////////////////////////////////////////////////
      // get reference value by looking at key in reference file
      //////////////////////////////////////////////////////////////////////
      char* valueFound;
      valueFound = findKeyValue(refer_data, row_elem[0]);
      cout << " value found - " << valueFound << endl;

      //////////////////////////////////////////////////////////////////////
      // output to file, first colmn replaced with value from reference file
      //////////////////////////////////////////////////////////////////////
      fout<<setw(10)      << valueFound
          <<setw(10)  << row_elem[4]
          <<setw(10)      << row_elem[3]
          <<setw(10)      << row_elem[2]
          <<setw(10)      << row_elem[1]
          <<endl;

      i=0;
}

return 0;
}
0
 
LVL 19

Expert Comment

by:drichards
ID: 12092672
1) Make sure you turn off precompiled headers for this file or add '#include "stdafx.h"' at the top of the file.

2) '     char *token = MyStrTok( row_read, "," );' should be:  char *token = MyStrTok( row_read, ',' );
Note the single quotes - my version of strtok takes a single character as a delimiter.  The code is a lot simpler if you don't need multiple delimiters.

If you need to turn off precompiled headers, right click on the source file in the Solution Explorer and select "Properties". Go to the C/C++->Precompiled Headers page and choose "Not Using Precompiled Headers" on the "Create/Use Precompiled Header" line.
0
 

Author Comment

by:justinY
ID: 12092893
drichards,
single quotes works, but other problems.
1. #include "stdafx.h" cannot be found in my vc7\include folder.
2. precompiled headers have been turn off.
3. without '#include "stdafx.h"', I can compile my code, but the output is funny. It only catch one column.
I post my fin, and frefer files. you run the code and will see the output.

fin:
id,notes1,notes2,notes3,account name,amount
111,aaa,a3,a4,,100000
222,bbb,b3,b4,,200000
333,ccc,c3,c4,cccc5,300000
444,ddd,d3,d4,dddd5,400000
555,eee,e3,e4,eeee5,500000
666,fff,f3,f4,ffff5,600000
777,ggg,g3,g4,gggg5,700000
888,hhh,h3,h4,hhhh5,800000
999,iii,i3,i4,iiii5,900000


frefer:
111,111aaa
222,222bbb
333,333ccc
444,444ddd
555,555eee
666,666fff
777,777ggg
888,888hhh
999,999iii


Thanks and waiting online
0
 
LVL 19

Accepted Solution

by:
drichards earned 500 total points
ID: 12093371
While parsing fin, you use MyStrTok to start but then use strtok in the loop.  Change:

       token = strtok( NULL, "," );
to
       token = MyStrTok( NULL, ',' );

in the 'while (fin.getline(...))' loop.

There is also a minor logic error in that you try to look up the header line info and get a result of 'None' which is prined out as the header if the ID column.  You'll want to fix that.  Probably the first time through (this will be the header line) don't do 'findKeyValue'.
0
 
LVL 55

Expert Comment

by:Jaime Olivares
ID: 12096767
Here is an implementation of strsep() function:
http://www.greatsnakes.com/Sepal/d4/d0/strsep_8c-source.html
0

Featured Post

Active Directory Webinar

We all know we need to protect and secure our privileges, but where to start? Join Experts Exchange and ManageEngine on Tuesday, April 11, 2017 10:00 AM PDT to learn how to track and secure privileged users in Active Directory.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Socket Programming (Unix) 8 142
c++ syntax question 9 52
Unable to start eclipse ? 17 154
No module found pypyodbc, 3 33
IntroductionThis article is the second in a three part article series on the Visual Studio 2008 Debugger.  It provides tips in setting and using breakpoints. If not familiar with this debugger, you can find a basic introduction in the EE article loc…
This article shows you how to optimize memory allocations in C++ using placement new. Applicable especially to usecases dealing with creation of large number of objects. A brief on problem: Lets take example problem for simplicity: - I have a G…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will be introduced to the member functions push_back and pop_back of the vector class. The video will teach the difference between the two as well as how to use each one along with its functionality.

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question