Solved

Best way to save and import data in a visual studio 2008 application

Posted on 2011-02-18
4
518 Views
Last Modified: 2012-08-14
Hi all

I'm looking for some advice on the best file format and import methods to use in my VB application. The sheer number of alternatives and the fact that I'm quite new to this type of programming is driving me nuts.

My application is business simulation that requires the administrator to input team decisions from printed forms. There are about 30 fields of data for each team and each team has a VB form to input the data. Once processed, the simulation generates reports and accounts for each team. This process is then repeated for each period in the simulation (up tom 12 periods)

I would like to store all the data for one simulation in one file and add to it when each period is played,b ut also keep the results from previous periods. I am thinking of storing the data from the VB forms in arrays as it is entered into the simulation software and then storing the results for each team into other arrays. These can then be displayed on the screen as tables. Then the whole lot would be written to a CSV file.

The CSV file would also contain summary data about that particular simulation (team names, current quarter, etc.) as well as the results of previous rounds.

My questions are:

Are arrays the best way to store the table variables?
Is a CSV file the best way to save this data to a file?
Does it matter in a CSV file if the rows do not all contain the same number of fields?
Would it be easier to use datagrids/tables instead of arrays?

If this is all too complicated, can anyone point me in the direction of a good article (or book) on how to process and file nlarge amounts of data.

I'd prefer not to use a database type application.

Any help greatly appreciated.

R|egards

Terry
0
Comment
Question by:Terrygordon
  • 2
4 Comments
 
LVL 96

Expert Comment

by:Bob Learned
ID: 34931480
I would take a look at JSON Serialization.  Javascript Object Notation is a compact serialization syntax, that doesn't have the overhead that XML suffers from.  The amount of code to serialize and deserialize makes this a good choice.

JSON Serialization
http://msdn.microsoft.com/en-us/library/bb410770.aspx
0
 
LVL 27

Accepted Solution

by:
MikeToole earned 500 total points
ID: 34933463
...Are arrays the best way to store the table variables?
Almost certainly not - if by that you mean to store 30 pieces of data as entries in any array and only know which is which by the value of the index.
Better to define object classes that describe the data more explicitly.  
A Simulation class could describe a simulation and contain a collection of Result class instances to store the data for a team for each period input.
The Result class would have 30 properties, one for each of the fields.
...Is a CSV file the best way to save this data to a file?
   Probably not, good as an interface/transfer mechanism, but too lightweight for this.
...Does it matter in a CSV file if the rows do not all contain the same number of fields?
   Yes, but you can leave fields blank (two consecutive commas)
...Would it be easier to use datagrids/tables instead of arrays?
No need for tables or datasets unless you're connecting to a backend database, and a datagrid is just the way to interact with data and isn't really connected to the storage method.

Using the object approach, the class instances are the in-memory store of data and acts as the source of data for display / analysis. These objects are then stored on disk as serialized versions.

You can use the .Net BinaryFormatter to convert an object to be stored into a file, I've attached a couple of routines that I used to Serialize/Unserialize objects.

 (JSON serialization (as suggested in the other response) would also probably do the job - I'm not familiar with it)

The approach I'd recommend is to define a class to describe the
   Assuming that the collection of fields always the same (with some possibly not filled-in) I would recommend
...There are about 30 fields of data for each team...



''' <summary>
        ''' Restore an object from a file containing the serialized version of it 
        ''' </summary>
        ''' <param name="Type">The data type for the current instance, used to construct the filename to store the serialized object</param>
        Friend Shared Function RestoreObject(ByVal [Type] As DataType) As Object
            With SaveFileInfo(Type)
                If .Exists Then
                    Dim binFmtr = New BinaryFormatter
                    Try
                        Dim fStream As Stream = New FileStream(.FullName, FileMode.Open)
                        Dim obj As Object = binFmtr.Deserialize(fStream)
                        fStream.Close()
                        Return obj
                    Catch ex As Exception
                        Utility.LogError(System.Reflection.MethodBase.GetCurrentMethod(), "Unable to restore saved data: " + ex.Message)
                        Return Nothing
                    End Try
                Else
                    Return Nothing
                End If
            End With
        End Function
        ''' <summary>
        ''' Serialize object and save to a file
        ''' </summary>
        ''' <param name="obj">The collection of data items to be saved</param>
        ''' <param name="Type"></param>
        ''' <param name="FileDate">The LastWriteTime of the imported file</param>
        ''' <remarks></remarks>
        Friend Shared Sub SaveObject(ByVal obj As Object, ByVal [Type] As DataType, ByVal FileDate As Date)
            Dim SaveFile = SaveFileInfo(Type)
            Dim fStream As New FileStream(SaveFile.FullName, FileMode.Create)
            'Construct a BinaryFormatter to serialize the data to the stream. 
            Dim binFmtr As New BinaryFormatter
            Try
                binFmtr.Serialize(fStream, obj)

            Catch e As SerializationException
                Utility.LogError(System.Reflection.MethodBase.GetCurrentMethod(), e.Message)
                Throw
            Finally
                fStream.Close()
                SaveFile.LastWriteTime = FileDate
                SaveFile.Refresh()
            End Try
        End Sub

Open in new window

0
 

Author Comment

by:Terrygordon
ID: 34934344
Thanks Mike. This makes things a lot clearer.
Regards

Terry
0
 
LVL 27

Expert Comment

by:MikeToole
ID: 34934459
Glad to help.
Sorry about the garbage paragraph at the end - it was work-in-progress and should've been deleted.
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to create and use a custom WaterMark textbox class.  The custom WaterMark textbox class allows you to set the WaterMark Background Color and WaterMark text at design time.   IMAGE OF WATERMARKS STEPS Create VB …
I think the Typed DataTable and Typed DataSet are very good options when working with data, but I don't like auto-generated code. First, I create an Abstract Class for my DataTables Common Code.  This class Inherits from DataTable. Also, it can …
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
This video shows how to quickly and easily add an email signature for all users on Exchange 2016. The resulting signature is applied on a server level by Exchange Online. The email signature template has been downloaded from: www.mail-signatures…

786 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question