Extracting a field from a data file

Posted on 2004-10-15
Last Modified: 2011-04-14
I am now writing a program to extract fields from a data.
Now I am experiencing problems to store the bytes read from the file into appropriate fields.
Since the file is not properly aligned, so I have to skip 0x09 and 0x0A character and 0x32 (which is space) as well everytime I read a byte from the file.

When I read the byte and pass it to the function to store in a field, the byte is not stored in appropriate fields since each field has different number of bytes.

I tried solving this problem by using
fstream extract_data;
extract_data << record.data1 << rercord.data2 << record.data2 ...
This works fine except when the particular record has a "blank" field.

What other options can I go for?
Question by:icysmarty
  • 4
  • 3
LVL 86

Expert Comment

ID: 12325630
That depends on what you want to shift your focus on. If you can easily change the data layout, introduce a special value that represents a blank (NULL) field. Or, change yur parsing code to cope with that. I'd go with the latter, especially if you e.g. know that two consecutive delimiters in fact do mean a blank field.

Author Comment

ID: 12325708
I don't get you.
What do you mean by two consecutive delimiters mean a blank field?

LVL 86

Expert Comment

ID: 12325737
Well, let's say you have

SomePerson|1978/07/21|$2000|A nice guy



you'd see that the fields re delimited by "|" and that "||" means "no content"...

Author Comment

ID: 12325752
Then how do I know if it is the end of field.
SomePerson|1978/07/21|$2000|A nice guy
Whoever|1978/07/21|$30000|A bad guy

The first field, SomePerson and Whoever have different length of field. How to detect the end of field, if I am reading it byte by byte from a data file?

For blank, I can keep a counter for the space, whenever I encounter more than 2 space, that means that field is a blank field right/
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

LVL 86

Accepted Solution

jkr earned 500 total points
ID: 12325758
>> How to detect the end of field

By looking for the delimiter, which is "|" in the above sample.

Expert Comment

ID: 12325803
Icysmart was suggesting that he was using the fstream >> operator, which to my knowledge always ignores whitespace (just like cin).  So, although your example of having | as the delimitor is good, I don't think it's the case here.

Icysmart, could you be more specific about what kind of input data you're reading from the file, and in what kind of variables you are trying to store it?

Author Comment

ID: 12325822
I decided not to use fstream >> operator cause it does not work out for my case.
I am reading a data file with 7 fields and storing them in a record of stuct type.
I create an array for each field and read the data file byte by byte.
The data file is like following

 40    john_smith   30,000 23 5
  50   xxyy.d     barbara     20,000 25 2

Obviously after 1st there will be a tab and line feed.. so I need to check for this conditions.. but there are too many conditions to be included in order to make the program runs perfectly. That's why I am asking if there are other c++ functions that I do not aware of.

Author Comment

ID: 12326526
>> By looking for the delimiter, which is "|" in the above sample

I got it. The program works. I didn't know that two delimiters means a blank field. Thank you very much.

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Many modern programming languages support the concept of a property -- a class member that combines characteristics of both a data member and a method.  These are sometimes called "smart fields" because you can add logic that is applied automaticall…
Container Orchestration platforms empower organizations to scale their apps at an exceptional rate. This is the reason numerous innovation-driven companies are moving apps to an appropriated datacenter wide platform that empowers them to scale at a …
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

25 Experts available now in Live!

Get 1:1 Help Now