Avatar of udanabanana
udanabananaFlag for Australia

asked on 

Reading a Binary / Structured / Formated File Using JAVA

Language: Java
OS: Windows XP
My Java knowledge: Learning/Low

I need to read a structured file(Hist.dat) which written using Visual Basic.
Visual Basic Structure is
        Type Histx
                Units(380) As Single ' Single is equal to float in java, 4 bytes
                Sell(380) As Single
                Cost(380) As Single
                Prom(380) As Integer ' Integer is equal to short in java, 2bytes
        End Type
so each record is (381 X 4) X 3 + 381 X 2 bytes , = 5334 bytes
it can be 100 000 records in a file. 5334 X 100 000 = 508.7 Mb

File Structure(a= first record, b= second record,  n= nth record)
Ua1....Ua381 Sa1......Sa381 Ca1....Ca381 Pa1.....Pa381 Ub1....Ub381 Sb1......Sb381 Cb1....Cb381 Pb1.....Pb381 ......................................Un1....Un381 Sn1......Sn381 Cn1....Cn381 Pn1.....Pn381

Ua1 = 4bytes representing first element of Unites in first record.
Pb381 = 2bytes repesenting last element of Prom in second record. and so on.
 You can download a sample Hist File from http://www.postecsolutions.com.au/EE/HisD5.zip (25KB)

Attention.
Keep in mind Hist.dat file can be 1Mb to 500 Mb.
Speed mabe a issue.
Is it possible to read a structured file written using VB by java? I assume it can be done.
__________________________________________________________
QUESTION:
I want to read this data from Hist.dat and write to comma seperated file as numbers. How can we do this? Whats the best way to read data from file? Please answer with a Full Working Example.
__________________________________________________________
Post Notes: I need a realy realy smart programmer to Answer this question. I post this question earlier and I didnt get the desired answer. I spend more than 10 days googling a answer. So kind frustrated now.

Please Please Help, Udana

JavaProgramming Languages-Other

Avatar of undefined
Last Comment
udanabanana
Avatar of Ajay-Singh
Ajay-Singh

Use Scanner (http://java.sun.com/j2se/1.5.0/docs/api/java/util/Scanner.html)
 
To read float, call nextFloat()
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Your data file (although > 26Mb) is completely empty of anything other than 0
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

Please Use This File If you need to.
 http://www.postecsolutions.com.au/EE/HisD6.zip
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

Ajay-Singh>> That wont Help. Have tried Everything I could. Thats why I'm asking a example.
Udana
Avatar of Jaax
Jaax
Flag of India image

Hi Udana,
  Few clarifications-
1. Is there a delimiter between each records ?
2. What does Units(380) signify ? Is it an array of 380 elements ?
3.Is it guaranteed that all the elements in your record will always be populated ?
Can it ever happen that any of Units or Prom can be null ?
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

Jaax.
1 Nope There are no delimiter. file is binary represnts of short and float arrays.
2 Units(380) means in VB array of 381 elements.
3 All elements not need to populated for each new or updatrd record. it can be null  or 0 bytes. However each record will take exactly 5334bytes in this case.
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

Sound Like This is Very Hard Question...
Few Tips for all the partcipating Experts..

1 My previous Question with similar issue>> https://www.experts-exchange.com/questions/22402467/How-to-read-sturctured-formated-binary-file-from-JAVA.html

2 Refer http://www.particle.kth.se/~lindsey/JavaCourse/Book/Part1/Java/Chapter09/fileBinaryIO.html
Regarding Binary I/O. Difference between this case and my case is this case each record has an Int and Float(two variables). In my case each record has 381 floats X 3 and 381 shorts(1524 variables).

3 I think We might need to use BinaryRead or File Object read.
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Why 381 and not 380?

Does that file have 3895 records by any chance
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

CHANGES.
I have two different set of files they either above mentioned file structure or
        Type PHist
                Units(55) As Single
                Sell(55) As Single
                Cost(55) As Single
                Prom(55) As Integer
        End Type
in this case record size is 784. These file could be this structure (784bytes)or previously mentioned structure. In this case following file is with this structure (784 bytes). These file structure change depend on our clients. I'm Sorry for the tangle. Anyway reading logic would be same.
 http://www.postecsolutions.com.au/EE/HisD6.zip
In this case file size would be 20,725,824 bytes and there for number of records would be  20,725,824 / 784. which is 26436.
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

CEHJ>>
I'm realy sorry for the inconvinience. I guess I have explain it above.
so its 26436 records.
in java
float[55] zzz; would be array zzz with 55 elements. (0,1,2,3,.....53,54)

in VB
Dim zzz(55) As Single means array zzz with high bound is 55. (0,1,2,3,4,......54,55) which is 56 elements
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

No problem. Here you go:

package demo;

import java.io.DataInput;
import java.io.DataInputStream;
import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.PrintStream;

public class Histx {
      private static int UNITS_SIZE, SELL_SIZE, COST_SIZE, PROM_SIZE;
      static {
            UNITS_SIZE = SELL_SIZE = COST_SIZE = PROM_SIZE = 56;
      }

      /**
       * @param args
       */
      public static void main(String[] args) {
            File f = new File("c:/install-files/HisD6.dat");
            DataInputStream in = null;
            int recCount = 0;
            try {
                  in = new DataInputStream(new FileInputStream(f));
                  while (Histx.printRecordAsCsv(in, System.out)) {
                        recCount++;
                  }
            } catch (FileNotFoundException e) {
                  e.printStackTrace();
            }
            finally {
                  if (in != null) {
                        try {
                              in.close();
                        } catch (IOException e) {
                              // Ignore
                        }
                  }
            }
            
      }

      public static boolean printRecordAsCsv(DataInput in, PrintStream out) {
            try {
                  for (int i = 0; i < Histx.UNITS_SIZE; i++) {
                        out.print(in.readFloat());
                        out.print(',');
                  }
                  for (int i = 0; i < Histx.SELL_SIZE; i++) {
                        out.print(in.readFloat());
                        out.print(',');
                  }
                  for (int i = 0; i < Histx.COST_SIZE; i++) {
                        out.print(in.readFloat());
                        out.print(',');
                  }
                  for (int i = 0; i < Histx.PROM_SIZE - 1; i++) {
                        out.print(in.readShort());
                        out.print(',');
                  }
                  out.print(in.readShort());
                  out.println();
                  return true;
            } catch (EOFException e) {
                  // Ignore
                  return false;
            } catch (IOException e) {
                  e.printStackTrace();
                  return false;
            }
      }
      
}
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

CEHJ>>
I tested it. WOW its work what a releif. Thanks mate. Why didnt I think like that. Well having said that. I was thinking rather than reading element at a time cant we read whole record at a time? Like can we deal single record as a object or so?
___________________________
In VB browsing through VB would be.
Open "c:/install-files/HisD6.dat" For Random Shared As #19 Len = 784
// open the file choped by 784 bytes(record length)    
Get #19, Recno, Hist
// geting the desired record by number(in this case Recno would be 1 to 26436)
// in here Hist refered as 'Type', earlier mentioned.
For i = 1 to 55
something = Hist.Units(i)
// so on
Next i
Close #19
// close the opened file
___________________

I guess you get the idea. I'm not comparing an eventdriven method with object oriented method, but in this case we can read the specific  record and alter if I need to. Out of curiousity I have a question for you.

1 Say we need to read record number 1234 and change 20th element of sell() array in 1234th record. How Can we do that?

-------------------------------------------------
In VB
Open "c:/install-files/HisD6.dat" For Random Shared As #19 Len = 784
 Hist.Sell(20) = whateveryouwant //(a single, of course)
Put#19, 1234, Hist
Close #19
//whole thing will take less than a milisecond or so..:)
--------------------------------------------------
ASKER CERTIFIED SOLUTION
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
Avatar of udanabanana
udanabanana
Flag of Australia image

ASKER

CEHJ>>
Im going to accept your answer.. no doubt about that. but  is it easy to read the record by record?
Anyway will ask the question again and will invite you to participate.
Udana.
Thank you very much for your Help.
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

:-)

It's easy yes but not light on resources. Got to go to bed now though ;-)
Java
Java

Java is a platform-independent, object-oriented programming language and run-time environment, designed to have as few implementation dependencies as possible such that developers can write one set of code across all platforms using libraries. Most devices will not run Java natively, and require a run-time component to be installed in order to execute a Java program.

102K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
Ask the experts
Read over 600 more reviews

TRUSTED BY

IBM logoIntel logoMicrosoft logoUbisoft logoSAP logo
Qualcomm logoCitrix Systems logoWorkday logoErnst & Young logo
High performer badgeUsers love us badge
LinkedIn logoFacebook logoX logoInstagram logoTikTok logoYouTube logo