Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Problem with StreamTokenizer

Posted on 2001-07-31
9
Medium Priority
?
623 Views
Last Modified: 2008-03-04
HI All,
I am reading a .csv file(comma separated) using stream tokenizer. My file is as the following:

1000,2000,"Inavesevich wins",5000
1001,2001,"Rafter loses",5001

I am reading it line by line and separating each line into tokens separated by comma.
If the file is in the above format I am not having any problem in reading.

If the file is in the following format:
1000,2000,Inavesevich wins,5000
1001,2001,Rafter loses,5001

i.e if the double quotes are removed from the String having spaces
I am having the problem. My program is taking "Ivanesevich" as one token and "wins" as another token whereas i want "Ivanesevich Wins" as a single token.

I am using the StreamTokenizer. My code goes as follows:

st.eolIsSignificant(true);
st.whitespaceChars(',',',');
st.quoteChar(',');

where st is the streamtokenizer.

How to avoid this problem?

If would be helpful if you can post some sample code.

TIA,
Benaam



0
Comment
Question by:benaam
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
9 Comments
 
LVL 92

Accepted Solution

by:
objects earned 400 total points
ID: 6336871
You could use StringTokenizer to do what you require.
Something like:

String line = null;
while (null!=(in.readLine()))
{
   StringTokenizer tokens = new StringTokenizer(line, ",\n\r");
   String a = tokens.nextToken();
   String b = tokens.nextToken();
   String c = tokens.nextToken();
   String d = tokens.nextToken();
   ...
}

0
 
LVL 4

Expert Comment

by:Neutron
ID: 6336890
You could try setting that space (' ') is an "ordinary character"

st.ordinaryChar( ' ' )

Greetings,
    Ntr:)
0
 
LVL 7

Expert Comment

by:tomboshell
ID: 6336894
The Stream Tokenizer is designed to seperate into tokens using a white space as the seperator.  You can add further tokens to it so that it will recognize these also as white-spaces, but it will still be white space driven.  To overcome that problem, go with objects suggestion and implement a stringTokenizer.  I agree that it would be your best bet.  It is a simpler tokenizer, located in the utils package.  As parameters pass the string to be parsed, and the deliminator.  There is also a boolean to set if you want it to return the token with the deliminator, seperated from the rest.  

Best of luck,
Tom
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 4

Expert Comment

by:Neutron
ID: 6336895
Anyway, it is better if your string is within quotes, because in that case string can contain comma literals.

Greetings,
    Ntr:)
0
 
LVL 3

Expert Comment

by:rjackman
ID: 6336906
hi
as object said it is better to use the StringTokenizer
try following

public void readAndParse()
{
  String file = "path of ur file";
  try
 {
  BufferedReader reader = new BufferedReader(new FileReader(new File(file)));
 Vector data = new Vector();
 String line = null;
 while((line=reader.readLine())!= null)
 {
    if(!line.trim().length<=0)
   {
     StringTokenizer tokenizer = new StringTokenizer  (line,",");
     while(tokenizer.hasMoreTokens())
    {
      data.add(tokenizer.nextToken());
    }

   }
 }
 }
 catch(Exception ioe)
 {
  ioe.printStackTrace();
 }
}

cheers
RJ
0
 
LVL 3

Expert Comment

by:rjackman
ID: 6336911
oops
chang following
if(!line.trim().length<=0)
to
if(line.trim().length()!=0)
cheers
RJ

0
 

Author Comment

by:benaam
ID: 6339621
Thanx Objects..This way it is working now..
0
 
LVL 1

Expert Comment

by:Moondancer
ID: 7024706
You have 16 open questions today which need your attention.  ADMINISTRATION WILL BE CONTACTING YOU SHORTLY.  Moderators Computer101, Netminder, Mindphaser or I will return to finalize these as soon as possible if after 7 days no response by the Asker.  EXPERTS-->  please post closing recommendations before that time.

Below are your open questions as of today.  Questions which have been inactive for 21 days or longer are considered to be abandoned and for those, your options are:
1. Accept a Comment As Answer (use the button next to the Expert's name).
2. Close the question if the information was not useful to you, but may help others. You must tell the participants why you wish to do this, and allow for Expert response.  This choice will include a refund to you, and will move this question to our PAQ (Previously Asked Question) database.  If you found information outside this question thread, please add it.
3. Ask Community Support to help split points between participating experts, or just comment here with details and we'll respond with the process.
4. Delete the question (if it has no potential value for others).
   --> Post comments for expert of your intention to delete and why
   --> YOU CANNOT DELETE A QUESTION with comments; special handling by a Moderator is required.

For special handling needs, please post a zero point question in the link below and include the URL (question QID/link) that it regards with details.
http://www.experts-exchange.com/jsp/qList.jsp?ta=commspt
 
Please click this link for Help Desk, Guidelines/Member Agreement and the Question/Answer process.  http://www.experts-exchange.com/jsp/cmtyHelpDesk.jsp

Click you Member Profile to view your question history and please keep them updated. If you are a KnowledgePro user, use the Power Search option to find them.  

Questions which are LOCKED with a Proposed Answer but do not help you, should be rejected with comments added.  When you grade the question less than an A, please comment as to why.  This helps all involved, as well as others who may access this item in the future.  PLEASE DO NOT AWARD POINTS TO ME.

To view your open questions, please click the following link(s) and keep them all current with updates.
http://www.experts-exchange.com/questions/Q.20034268.html
http://www.experts-exchange.com/questions/Q.20063886.html
http://www.experts-exchange.com/questions/Q.20071336.html
http://www.experts-exchange.com/questions/Q.20132970.html
http://www.experts-exchange.com/questions/Q.20152566.html
http://www.experts-exchange.com/questions/Q.20160660.html
http://www.experts-exchange.com/questions/Q.20161367.html
http://www.experts-exchange.com/questions/Q.20161370.html
http://www.experts-exchange.com/questions/Q.20166906.html
http://www.experts-exchange.com/questions/Q.20166907.html
http://www.experts-exchange.com/questions/Q.20181828.html
http://www.experts-exchange.com/questions/Q.20182284.html
http://www.experts-exchange.com/questions/Q.20185356.html

To view your locked questions, please click the following link(s) and evaluate the proposed answer.
http://www.experts-exchange.com/questions/Q.20036734.html
http://www.experts-exchange.com/questions/Q.20161368.html
http://www.experts-exchange.com/questions/Q.20161395.html

*****  E X P E R T S    P L E A S E  ******  Leave your closing recommendations.
If you are interested in the cleanup effort, please click this link
http://www.experts-exchange.com/jsp/qManageQuestion.jsp?ta=commspt&qid=20274643 
------> POINTS FOR EXPERTS awaiting comments are listed in the links below
http://www.experts-exchange.com/commspt/Q.20277028.html (Part 1)
http://www.experts-exchange.com/jsp/qShow.jsp?ta=commspt&qid=20295853 (Part 2)
 
Moderators will finalize this question if in @7 days Asker has not responded.  This will be moved to the PAQ (Previously Asked Questions) at zero points, deleted or awarded.
 
Thanks everyone.
Moondancer
Moderator @ Experts Exchange
0
 
LVL 5

Expert Comment

by:Netminder
ID: 7329724
Question abandoned; force-accepted.

Netminder
CS Moderator
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Viewers will learn about the different types of variables in Java and how to declare them. Decide the type of variable desired: Put the keyword corresponding to the type of variable in front of the variable name: Use the equal sign to assign a v…
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
Suggested Courses

604 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question