Expiring Today—Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Reading mixed data from a text file.

Posted on 2003-11-20
7
Medium Priority
?
371 Views
Last Modified: 2010-05-01
I wish to read data from a text file that can vary in size from a few bytes to a few hundred Megabytes in size.
The data is rather "mixed". There will be two types of lines.. data and occasional time stamps..

Each line of data will have 9 data values separated by a comma and ending with a CR/LF.
Each time stamp line will be just a series of bytes indicating Day of Week, Year, Month... etc...

So looking at the file, you will see a time stamp line, followed by several lines of data associated with that time stamp... followed by another time stamp, and more data.. etc..

Here is an example of what the file looks like:
----------------------------------------------------------
4031119112557                                                          << time stamp
0,7228,19942,29396,36720,40631,41293,60838,0
0,7233,16744,27259,33305,39699,41602,60837,1          << data
0,7222,17645,28492,35510,39312,38306,60848,2
0,7229,16677,27185,35742,37983,38328,60842,3
4031119114816                                                             << another time stamp
400,13692,25187,34470,37555,39239,40761,60224,0
0,7162,19017,28989,37766,40992,41219,60246,1              << more data
0,7143,24054,30181,36815,39571,40076,60251,2
0,7140,21018,29885,35846,38206,39768,60267,3
0,7153,23132,31717,37398,40756,41971,60227,4
0,7154,18958,29391,37196,39542,40636,60223,5
0,7152,20048,29692,37307,39751,41080,60224,6
0,7136,20733,32039,37401,40820,42348,60238,7
... etc..
---------------------------------------------------------------
format for the time stamp is "DYYMMDDHHMMSS"  with the first D being the nth day of the week

This can go on for a long time... there may long intervals between time stamps or short.. it will vary..

I'm trying to do several things with this data... I need to read all the data (ignoring the time stamps) so I can just store the raw data.. I don't know how to just ignore time stamp lines...
I also would like (as an option) to somehow distinguish each piece of data with an appropriate day, or month.. (taken from the timestamp)..  so this would involve reading the time stamp, putting it into a group (group by day or by month).. then storing all the data lines associated with those times stamps..

The overal goal of all this is to have this data, which is taken at various times during a day..
I don't know if this is confusing, I hope it's clear.
I appreciate any help.

Jeff
0
Comment
Question by:jscharpf
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 2
7 Comments
 

Author Comment

by:jscharpf
ID: 9788119
I wanted to add.. the data must be kept in the order it is shown, and sometimes there are timestamps without any data following. (you'll just see two time stamps one after the other) I can ignore these...

Jeff
0
 
LVL 43

Accepted Solution

by:
TimCottee earned 2000 total points
ID: 9788169
Hi jscharpf,

Basically you need to scan it like this:

Dim intFHandle As Integer
intFHandle = FreeFile()
Dim strLine As String
Dim aryElements() As String
Open "Myfile.txt" For Input As #intFHandle
Do
  Line Input #intFHandle,strLine
  If Instr(strLine,",") Then 'This is an element line and not a timestampe
    aryElements = Split(strLine,",") 'break up the line into an array we can then handle
    For i = 0 To 8
      txtDisplay(i).Text = aryElements(i) 'Show the result in a text box (one for each element)
    Next
  Else
    'It is a timestamp so do something else with it for example:
    Msgbox "Timestamp of : " & vbLf & Mid(strLine,6,2) & "/" & Mid(strLine,4,2) & "/" & Mid(strLine,2,2) 'dd/mm/yy format display
  End If
Loop Until Eof(intFHandle)
Close #intFHandle

This of course is just an example of how you could process the file, the exact operations on each line would be up to you.

Tim Cottee MCSD, MCDBA, CPIM
Brainbench MVP for Visual Basic
http://www.brainbench.com
0
 
LVL 86

Expert Comment

by:Mike Tomlinson
ID: 9788716
What is the end goal here?

Do you want an easier way to view data sets associated with a time stamp?
... or are you looking to do some calculations based upon the sets and only look at the results?
... or are you wanting to see timestamps with data sets and some kind of calculated results together?

Idle_Mind
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 

Author Comment

by:jscharpf
ID: 9788864
There won't be an calculations on the data. It will only be graphed. But the problem I have is that the data will be taken at various times during a day, and for undefined lengths of time. So, for example, A time stamp of 11 Am on a certain day wil be followed by a bunch of data.. I'm not sure how many. It could be zero, one, or 10000.... then, another time stamp, followed by more data.. these will be stamped at various times during the day (

To give you more info.. I'm installing a microcontroller in a power tool and letting the operator use it. Each time the operator turns it on and uses it, I time stamp it, then begin saving data until it's turned off. This can take place several times in a day. We are limited in our data saving options for now.. this is how we are getting the data..

So.. I will graph (I have a very good OCX for this) all 8 data points vs. time, and initially this will just be one long graph, ignoring the time stamps, and knowing the sample rate is XX ms, we will see a profile and we can look at all the data..
but as an added option, we will need to somehow ALSO (I can do two graphs and two sets of data if I need to) need to somehow associate what days each part of the graph correspond to.. The graphing OCX I have is very good and can graph an unlimited set of points, so I'm thinking of just storing a second set of data, that is associated with the time stamps, then giving the user the option to just go day by day and look at data for that day only.. (redrawing the graph with the new data each time)..

I am sorry if this is confusing.. :)

Jeff
0
 

Author Comment

by:jscharpf
ID: 9788905
also.. lol..
suppose the operator turns the tool on and off 100 times in one day.. I will have alot of data and alot of time stamps to deal with. I'm just trying to find a good way to group these so that I don't have a million graphs (one for each time stamp would be crazy)
 :)
I realize that I have to compromise somewhere.. so I thought that maybe grouping the data by Day... and just keeping sets of data per day.. it might help.

BUt the MOST important part is to get all the data out and separate it from the timestamps.. so I can at least see nothing but data. I can worry about when it happened later.. Doing this at least gives us good info, we just don't know when it occured (day, etc.)

Jeff
0
 
LVL 86

Expert Comment

by:Mike Tomlinson
ID: 9789123
One approach would be to read the file in just as TimCottee has shown you and add the arrays of data to a collection.

If you want to get tricky, then you could create a separate collection for each day, then have a collection of collections.

As you read each stamp, convert it to a date, without the time data in it, to be used as a key.  Then add all data you read into a collection until you reach a stamp that is from the next day.  Now add the collection you just made to the main collection using the date as a key, and then start a new collection that will have the new stamp key associated with it.

You could place all the date keys in a listbox so you can easily select a different day to view and use the selected item text to retrieve the correct collection containing the corresponding data.

I hope that wasn't too terribly confusing.

Idle_Mind

0
 

Author Comment

by:jscharpf
ID: 9796483
Thanks to both of you for your answers! I guess I can only give points to one person.


Jeff
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When trying to find the cause of a problem in VBA or VB6 it's often valuable to know what procedures were executed prior to the error. You can use the Call Stack for that but it is often inadequate because it may show procedures you aren't intereste…
Article by: Martin
Here are a few simple, working, games that you can use as-is or as the basis for your own games. Tic-Tac-Toe This is one of the simplest of all games.   The game allows for a choice of who goes first and keeps track of the number of wins for…
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…
Suggested Courses

719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question