asked on

Java parsing, better way to use tokenizer?

I'm trying to parse a csv file (probably made in excel, or access).

Each field is separated by a "," (comma) but some fields are empty, so I'm not getting the data correctly.

Below is what I'm doing now. Is there a better way to parse?

This line works fine:
23, something, aTitle, something, Smith

This line gives me wrong values:

21,,anotherTitle,,Jones

Both come from the same table with 4 fields.

Thanks.

//...			
String line = in.readLine();
while(line!=null)
{
	StringTokenizer st = new StringTokenizer(line,",");
	int fieldCount=0;
			
	while(st.hasMoreTokens())
	{
		String nextFieldData = st.nextToken();
		++fieldCount;
				
					
		switch(fieldCount)
		{
			case 1: 
			String ID = (nextFieldData);
			break;
						
			case 3: 
			String Title  = (nextFieldData);
			break;
 
			case 5: 
			String Name  = (nextFieldData);
			break;
		}
//...

Open in new window

ksivananth

there are lot of issues which you have to take care if you do it that ways...

instead try some readily available parsers,

http://opencsv.sourceforge.net/
http://www.csvreader.com/

polkadot

ASKER

csvreader works great, but its a bit bulky, I just wanted some ideas in just parsing it simply ... any other ideas

mrcoffee365

Then you can't use StringTokenizer, you have to write your own parser. Read the line character by character, check to see if there's a comma, and parse accordingly.

CEHJ

http://ostermiller.org/utils/CSV.html

I would not use home-brewed parsing. If it were that simple, there would not be any need for classes such as these

ksivananth

>>StringTokenizer st = new StringTokenizer(line,",");

to

StringTokenizer st = new StringTokenizer( line,",", true );

and discard the odd( which is the comma ) tokens in the traversing process...

Mick Barry

you can find a lightweight csv parser here:

http://mindprod.com/zips/csv24.zip

ASKER CERTIFIED SOLUTION

gnoon

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

gnoon

public String[] parse(String s, char delim)
{
StringBuffer b = new StringBuffer();
ArrayList a = new ArrayList();
char c;
for(int i=0; i<s.length(); i++)
{
c = s.charAt(i);
if(c == delim)
{
a.add(b);
b = new StringBuffer();
}
else b.append(c);
}
String[] r = new String[a.size()];
for(int i=0; i<r.length; i++)
r[i] = (String) a.get(i);
return r;
}