Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

Java parsing, better way to use tokenizer?

Posted on 2007-11-23
10
Medium Priority
?
1,030 Views
Last Modified: 2008-09-20
I'm trying to parse a csv file (probably made in excel, or access).

Each field is separated by a "," (comma) but some fields are empty, so I'm not getting the data correctly.

Below is what I'm doing now. Is there a better way to parse?


This line works fine:
23, something, aTitle, something, Smith

This line gives me wrong values:

21,,anotherTitle,,Jones

Both come from the same table with 4 fields.

Thanks.
//...			
String line = in.readLine();
while(line!=null)
{
	StringTokenizer st = new StringTokenizer(line,",");
	int fieldCount=0;
			
	while(st.hasMoreTokens())
	{
		String nextFieldData = st.nextToken();
		++fieldCount;
				
					
		switch(fieldCount)
		{
			case 1: 
			String ID = (nextFieldData);
			break;
						
			case 3: 
			String Title  = (nextFieldData);
			break;
 
			case 5: 
			String Name  = (nextFieldData);
			break;
		}
//...

Open in new window

0
Comment
Question by:polkadot
8 Comments
 
LVL 26

Expert Comment

by:ksivananth
ID: 20339104
there are lot of issues which you have to take care if you do it that ways...

instead try some readily available parsers,

http://opencsv.sourceforge.net/
http://www.csvreader.com/
0
 

Author Comment

by:polkadot
ID: 20339344
csvreader works great, but its a bit bulky, I just wanted some ideas in just parsing it simply ... any other ideas
0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 20339554
Then you can't use StringTokenizer, you have to write your own parser.  Read the line character by character, check to see if there's a comma, and parse accordingly.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 86

Expert Comment

by:CEHJ
ID: 20339614
http://ostermiller.org/utils/CSV.html

I would not use home-brewed parsing. If it were that simple, there would not be any need for classes such as these
0
 
LVL 26

Expert Comment

by:ksivananth
ID: 20339620
>>StringTokenizer st = new StringTokenizer(line,",");

to

StringTokenizer st = new StringTokenizer( line,",", true );

and discard the odd( which is the comma ) tokens in the traversing process...
0
 
LVL 92

Expert Comment

by:objects
ID: 20340802
you can find a lightweight csv parser here:

http://mindprod.com/zips/csv24.zip
0
 
LVL 16

Accepted Solution

by:
gnoon earned 2000 total points
ID: 20344564
I've surveyed the source code of ST. If delemeters are successive (having blank field(s)), they will be group as a delemeter. For example

data: 1,,3,   will become   1,3,   (ST acts with ,, as ,)
return: {1,3,}

ST is not appropriate to parse CSV file.
If you will use JRE 1.4+, use String.split() instead    String[] fields = line.split(",").
If you will use JRE 1.3-, write your own function to parse the line (never use StringTokenizer) and returns array of fields.
0
 
LVL 16

Expert Comment

by:gnoon
ID: 20344587
public String[] parse(String s, char delim)
{
    StringBuffer b = new StringBuffer();
    ArrayList a = new ArrayList();
    char c;
    for(int i=0; i<s.length(); i++)
    {
        c = s.charAt(i);
        if(c == delim)
        {
            a.add(b);
            b = new StringBuffer();
        }
        else b.append(c);
    }
    String[] r = new String[a.size()];
    for(int i=0; i<r.length; i++)
        r[i] = (String) a.get(i);
    return r;
}
0

Featured Post

Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction This article is the first of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article explains our test automation goals. Then rationale is given for the tools we use to a…
In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
Suggested Courses
Course of the Month14 days, 23 hours left to enroll

577 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question