• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 277
  • Last Modified:

Easy Que: How to load/import .dat file in fastest way?

I am working on a project, which takes input data from .dat file which is 3000 charaters/row and more than 30000 rows per file.

- There is not delimiter in .dat file. Each field is of fixed number of charaters and fixed poisition in row.
- If there is no data for field then filled with blank spaces.
- Field's data may be alphanumaric

Which one is fastest way to import data? and how

1. Into predefined 'TYPE' structure?
2. Into CLASS?
3. Into predefined charater 'ARRAY'?

Please give me example
  • 5
  • 2
1 Solution
It does not matter which structure you put your data into,
the important part is loading your file.

and since I think you are talking about a Text file, then I except loading would be extremely fast.

Really do not suggest using a Type. since for high volume of data it vcan be troublesome.

Best neat method would be a class.

how big is your file?

sanjuoAuthor Commented:
Thanks for response. The file size may vary from >100mb to gigs. Can you give me example?
you said there is no delimiter in your file .

In this case you need to read fie line by line

Open "C:\Myfile.Dat" for input  as #1

dim s
line inplut #1,s

'now s is 1st line of your file
'name is 1st filed and defined as 7 charctets 1-7
dim Name

'Get 1st value  supoose its is is columns 1-6

Name =mid(s,1,7)
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

Do you know how to build and use a class?
sanjuoAuthor Commented:
if DAT file format is like this

1-4      4      ADMINISTRATION-DATE(mmyy)
5-6      2      LEVEL-CODE
7-8      2      REGION NUMBER
9-17      9      CAMPUS NUMBER
18-32      15      DISTRICT-NAME
33-47      15      CAMPUS-NAME
48-62      15      LAST-NAME
63-72      10      FIRST-NAME
73-73      1      MIDDLE-INITIAL
74-82      9      USER-ID
83-83      1      SEX-CODE(M/F)

then how do you create class and make sure correct size for each field. Because I did not find how to declare string field size for property in class builder.

Also how to use collection in this case?
Its good that you know how to use class builder.

now , class builder does not do data validations  ex:

Option Explicit
Private mvarfname As String 'local copy

Public Property Let fname(ByVal vData As String)

    mvarfname = vData
End Property

Public Property Get fname() As String
'used when retrieving value of a property, on the right side of an assignment.
'Syntax: Debug.Print X.fname
    fname = mvarfname
End Property

you need to validate it yourself

Public Property Let fname(ByVal vData As String)

   if len(vdata)>10   then
     "msgbox invalid name"
      exit property
   end if
    mvarfname = vData


End Property

Also how to use collection in this case?

After creating all properties in your main code you use this:

Dim c as new myclass
c.add  ADMINISTRATION-DATE,LEVEL-CODE,REGION NUMBER,.....   'using actual data you have read from thetext file

you should not use the collection inside the class directly.

The class itself sholud be used.

after ading all rows  you can retrieve them like this:

dim aDDMIN as Date

and so on...
My comments were a good answer to the question.

An accept with Grade A or B is proper.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

The 14th Annual Expert Award Winners

The results are in! Meet the top members of our 2017 Expert Awards. Congratulations to all who qualified!

  • 5
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now