Solved

Easy Que: How to load/import .dat file in fastest way?

Posted on 2004-04-12
10
236 Views
Last Modified: 2008-02-26
I am working on a project, which takes input data from .dat file which is 3000 charaters/row and more than 30000 rows per file.

- There is not delimiter in .dat file. Each field is of fixed number of charaters and fixed poisition in row.
- If there is no data for field then filled with blank spaces.
- Field's data may be alphanumaric

Which one is fastest way to import data? and how

1. Into predefined 'TYPE' structure?
2. Into CLASS?
3. Into predefined charater 'ARRAY'?

Please give me example
0
Comment
Question by:sanjuo
  • 5
  • 2
10 Comments
 
LVL 15

Expert Comment

by:unknown_routine
ID: 10806146
It does not matter which structure you put your data into,
the important part is loading your file.

and since I think you are talking about a Text file, then I except loading would be extremely fast.


Really do not suggest using a Type. since for high volume of data it vcan be troublesome.


Best neat method would be a class.

how big is your file?

0
 

Author Comment

by:sanjuo
ID: 10806496
Thanks for response. The file size may vary from >100mb to gigs. Can you give me example?
0
 
LVL 15

Expert Comment

by:unknown_routine
ID: 10806568
you said there is no delimiter in your file .

In this case you need to read fie line by line


Open "C:\Myfile.Dat" for input  as #1

dim s
line inplut #1,s

'now s is 1st line of your file
'name is 1st filed and defined as 7 charctets 1-7
dim Name

'Get 1st value  supoose its is is columns 1-6

Name =mid(s,1,7)
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 15

Expert Comment

by:unknown_routine
ID: 10806811
Do you know how to build and use a class?
0
 

Author Comment

by:sanjuo
ID: 10807608
if DAT file format is like this

RANGE      SIZE      NAME
--------------------------------------------------------------
1-4      4      ADMINISTRATION-DATE(mmyy)
5-6      2      LEVEL-CODE
7-8      2      REGION NUMBER
9-17      9      CAMPUS NUMBER
18-32      15      DISTRICT-NAME
33-47      15      CAMPUS-NAME
48-62      15      LAST-NAME
63-72      10      FIRST-NAME
73-73      1      MIDDLE-INITIAL
74-82      9      USER-ID
83-83      1      SEX-CODE(M/F)

then how do you create class and make sure correct size for each field. Because I did not find how to declare string field size for property in class builder.

Also how to use collection in this case?
0
 
LVL 15

Accepted Solution

by:
unknown_routine earned 125 total points
ID: 10807820
Its good that you know how to use class builder.

now , class builder does not do data validations  ex:


Option Explicit
Private mvarfname As String 'local copy

Public Property Let fname(ByVal vData As String)

    mvarfname = vData
End Property


Public Property Get fname() As String
'used when retrieving value of a property, on the right side of an assignment.
'Syntax: Debug.Print X.fname
    fname = mvarfname
End Property



you need to validate it yourself


Public Property Let fname(ByVal vData As String)

   if len(vdata)>10   then
     "msgbox invalid name"
      exit property
   end if
    mvarfname = vData

 

End Property


Also how to use collection in this case?

After creating all properties in your main code you use this:

Dim c as new myclass
c.add  ADMINISTRATION-DATE,LEVEL-CODE,REGION NUMBER,.....   'using actual data you have read from thetext file

you should not use the collection inside the class directly.

The class itself sholud be used.


after ading all rows  you can retrieve them like this:

dim aDDMIN as Date
aDDMIN=c.item(1).ADMINISTRATION-DATE

and so on...
0
 
LVL 15

Expert Comment

by:unknown_routine
ID: 11258928
My comments were a good answer to the question.

An accept with Grade A or B is proper.

Regards.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
DIR issue 7 51
Concatenate 2 userform comboboxes to a userform text box 4 56
Visual Studio search word table and return Cell index 8 49
Add and format columns in vb6 7 47
Have you ever wanted to restrict the users input in a textbox to numbers, and while doing that make sure that they can't 'cheat' by pasting in non-numeric text? Of course you can do that with code you write yourself but it's tedious and error-prone …
If you have ever used Microsoft Word then you know that it has a good spell checker and it may have occurred to you that the ability to check spelling might be a nice piece of functionality to add to certain applications of yours. Well the code that…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

24 Experts available now in Live!

Get 1:1 Help Now