Solved

using an index file to print out specific records

Posted on 2003-10-30
19
446 Views
Last Modified: 2010-03-31
Hi im trying to print out some records from a raf file firstly in the order they are in, in the file then in order that they are according to the index file and Im pretty sure I have the right code. IT prints them out for the first part fine but when it comes to print them out according to the index file it says

Exception in thread "main" java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
at java.io.DataInputStream.readUTF(DataInputStream.java:580)
at java.io.RandomAccessFile.readUTF(RandomAccessFile.java:857)
at ShowDatabase.main(ShowDatabase.java:50)
line 50 is System.out.println(raf.readUTF()); down the bottom


public class ShowDatabase
{
static final int recLength = 113;

public static void main (String [] args ) throws Exception
{


RandomAccessFile raf = new RandomAccessFile("drivers.dbf","rw");
int numRecs = (int) raf.length()/recLength;

for (int x = 0; x < numRecs; x++)
{
System.out.println(raf.readUTF());
System.out.println(raf.readInt());
raf.readUTF();
raf.readChar();
System.out.println(raf.readInt());
}


File f = new File("license.ndx");
FileInputStream fIn = new FileInputStream(f);
DataInputStream dIn = new DataInputStream(fIn);

int offset[] = new int[numRecs];
int license[] = new int[numRecs];


for (int y = 0; y < numRecs; y++)
{
license[y] = dIn.readInt();
offset[y] = dIn.readInt();
}
fIn.close();

for(int z=0; z<numRecs; z++)
{

raf.seek((int)offset[z]);

System.out.println(raf.readUTF());
System.out.println(raf.readInt());
raf.readUTF();
raf.readChar();
System.out.println(raf.readInt());

}

raf.close();

}
}
0
Comment
Question by:bella2k4
  • 11
  • 8
19 Comments
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
Add the following line before the "raf.see((int)offset[z]);"

System.out.println("offset[" + z + "] = " + offset[z]);

This will tell you the offset that it thinks record z is at.

It looks like you are trying to read beyond the end of the file.  You probably need to check your indices to make sure you are "seek"ing to the location you really want.

Also, if you are only reading the file, change the "rw" in the RandomAccessFile constructor to "r" for safety ;-)
0
 

Author Comment

by:bella2k4
Comment Utility
offset[0] = 162

and the license index file contains
license number - index
------------------------------

696458  162
1504503  243
1519980  567
3501227  486
3510402  81
3513671  0
3513986  405
3516037  324
0
 

Author Comment

by:bella2k4
Comment Utility
I figured out the problem its something to do with me trying to create an index based on firstname + " " + surname it doesnt like.

originally i was doing it like

         surname[x] = seqfile.readUTF().trim();
          firstname[x] = seqfile.readUTF().trim();
          name[x] = firstname[x] + " " + surname[x];      


and then


//writing the random access file
      for(int y=0; y<numRecs;++y)
      {
          raf.writeUTF(name[y].concat("                               ").substring(0,31) );

//31 being firstname 15 + surname 15 + 1 for spacing ?

          raf.writeInt(license[y]);
          raf.writeUTF(address[y].concat("                                    ").substring(0,35) );
          raf.writeChar(gender[y]);
          raf.writeInt(yearIssued[y]);
      }

Can anyone see anything wrong
0
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
I've built the system (I had to create some code to write the files ;-))

What I have found is that the entries are 80 bytes long, so my indices are:

0, 80, 160, 240 etc...

In your ndx file, they are 1 greater.

This means that when you try to read the UTF string, it is starting one byte further down the stream than it should, so the length of the UTF string is being corrupted and it is trying to read too many bytes.

The problem is in the way you create your .ndx file.  Your offset is 1 too big.
0
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
If you want to check, here is the code I used to create the drivers.db and license.ndx files with test data:

Your existing ShowDatabase class should read the results of this OK.

BE CAREFUL.  BACKUP ANY DATA YOU WANT TO KEEP FIRST ;-)

import java.io.*;

public class CreateDatabase
{
    private void init()
    {
        String[] name = new String[20];
        int[] license = new int[20];
        String[] address = new String[20];
        char[] gender = new char[20];
        int[] yearIssued = new int[20];
       
        for (int i = 0; i < 20; i++)
        {
            name[i] = "name " + i;
            license[i] = 1234560 + i;
            address[i] = "lives at " + i;
            gender[i] = 'M';
            yearIssued[i] = 1985;
        }

        try
        {
            RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
            DataOutputStream dos = new DataOutputStream(new FileOutputStream("license.ndx"));
            for (int y = 0; y < 20; ++y)
            {
                raf.writeUTF(name[y].concat("                               ").substring(0, 31));
                raf.writeInt(license[y]);
                raf.writeUTF(address[y].concat("                                    ").substring(0, 35));
                raf.writeChar(gender[y]);
                raf.writeInt(yearIssued[y]);
               
                dos.writeInt(license[y]);
                dos.writeInt((int)raf.getFilePointer());
            }
           
            dos.close();
            raf.close();
        }
        catch (IOException ioe)
        {
            System.err.println("IOE: " + ioe.toString());
        }
    }

    public static void main(String[] args)
    {
        new CreateDatabase().init();
    }
}
0
 

Author Comment

by:bella2k4
Comment Utility
          surname        String  17 (2+15)
           firstname      String 17 (2+15)
           name[]         String + " " + String
           gender         char     2 (Unicode character)
           address        String  37 (2+35)
           yearIssued     int      4
           licenseNumber  Int      4
                       
       Total record length = 81 characters

So im trying to base an index not only on license number but name which is firstname and surname,  when you concatenate them is it 15 + 15 + 1 space (" ")

like for the name something like.

 firstname[i] = "firstname " + i;
 surname[i] = "surname " + i;
 fullname[i] = firstname[i] + " " + surname[i];

raf.writeUTF(fullname[y].concat("                               ").substring(0, 31));

0
 
LVL 15

Accepted Solution

by:
jimmack earned 125 total points
Comment Utility
I'm not sure if you are confirming what I've done or whether you are asking another question.

You created the file with the following writes (quoted from your earlier post with line numbers added by me):

//writing the random access file
      for(int y=0; y<numRecs;++y)
      {
1)          raf.writeUTF(name[y].concat("                               ").substring(0,31) );

//31 being firstname 15 + surname 15 + 1 for spacing ?

2)          raf.writeInt(license[y]);
3)          raf.writeUTF(address[y].concat("                                    ").substring(0,35) );
4)          raf.writeChar(gender[y]);
5)          raf.writeInt(yearIssued[y]);


1) a 31 character UTF String = 33 bytes
2) an int = 4 bytes
3) a 35 character UTF String = 37 bytes
4) a char = 2 bytes
5) an int = 4 bytes

33 + 4 + 37 + 2 + 4 = 80.

It doesn't really matter how the name is put together, you still only write the first 31 characters (using substring).

The .ndx file is just pairs of ints accessed using a DataInputStream, so lengths of license number doesn't really matter.

I really think the key is the size of the offset that you are calculating when you create the .ndx file.  Is there any chance you could post the code for that?
0
 

Author Comment

by:bella2k4
Comment Utility
oh i see what your getting at, i had the record length as if i was using first name and surname in a non concatenated form, so it will be 80, my bad.

Ill test it out see what happens
0
 

Author Comment

by:bella2k4
Comment Utility
You fixed it well done,

firstly when im displaying according to name index file and license index file they seem to be in the exact same order. could just be a coinsidence that they work out to be the same order.

Secondly, checkout the offset values when displaying according to name index.


//straight out of the raf file

Alison M Chappell
696458
1963

Barbara J Furza
1504503
1962

//displayed according to license index

offset[0] = 160
Glen K Lewis
1519980
1943

offset[1] = 240
Maureen E Dixon
3501227
1949


//sorted according to name index

offset2[0] = 1769172846
Glen K Lewis
1519980
1943

offset2[1] = 1751216240
Maureen E Dixon
3501227
1949

0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 

Author Comment

by:bella2k4
Comment Utility
nevermind found the problem
0
 

Author Comment

by:bella2k4
Comment Utility
now I got this problem again

when it goes to use the name index

--------------------------------------------------
offset2[0] = 1769172846
Exception in thread "main" java.io.EOFException
        at java.io.RandomAccessFile.readUnsignedShort(RandomAccessFile.java:640)

        at java.io.DataInputStream.readUTF(DataInputStream.java:574)
        at java.io.RandomAccessFile.readUTF(RandomAccessFile.java:857)
        at ShowDatabase.main(ShowDatabase.java:86)
0
 

Author Comment

by:bella2k4
Comment Utility
fixed it, i was reading in name as a int
me.idiot = true
0
 

Author Comment

by:bella2k4
Comment Utility
new error
ive tried to append a record to the end of the database file and then display it

getting this error
last record -
-----------------------------------

jon doe
3843212
1908

------------------------------------
Exception in thread "main" java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:397)
        at ShowDatabase.main(ShowDatabase.java:43)
0
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
I got the same error, but on the other file :-)

In my case, it's the RandomAccessFile that's running off the end.

In your case, did you update the index file with the appropriate data as well as the db file?
0
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
OK.  I've got it ;-)

I now have three files (and I've included the source for all of the below).  I can also confirm that these files are tested and work ;-)

I have made the following changes (some of which you will have already done, some of which might be relevant to your problem):

ShowDatabase.java
1) Changed recLength constant to 80

CreateDatabase.java
1) Added a variable called "startOfRecord" before the "for(int y" loop, to keep track of the real offset of the start of this record (previously, I was using the offset *after* the record had been written, ie. the start of the *next* record :-()
2) Use this variable when writing to the .ndx file in the same loop.
3) Changed the ++y in the loop to y++ to make sure that y is not incremented to 1 for the first record (which should be 0).

AppendDatabase.java  (new file)
The class does a "seek" to the end of the db file in order to write to it.  It also updates the .ndx file.  NOTE:  When opening the .ndx file for writing, the second parameter in the constructor is "true", meaning - append to file.
Again, as with changes 1) and 2) for CreateDatabase.java, the recordPointer is stored before the db file is updated, then this value is appended to the .ndx file.

I hope this helps ;-)

Jim.


ShowDatabase.java
---------------------------
import java.io.*;

public class ShowDatabase
{
    final static int recLength = 80;

    public static void main(String[] args) throws Exception
    {

        RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
        int numRecs = (int)raf.length() / recLength;

        for (int x = 0; x < numRecs; x++)
        {
            System.out.println(raf.readUTF());
            System.out.println(raf.readInt());
            raf.readUTF();
            raf.readChar();
            System.out.println(raf.readInt());
        }

        File f = new File("license.ndx");
        FileInputStream fIn = new FileInputStream(f);
        DataInputStream dIn = new DataInputStream(fIn);

        int offset[] = new int[numRecs];
        int license[] = new int[numRecs];

        for (int y = 0; y < numRecs; y++)
        {
            license[y] = dIn.readInt();
            offset[y] = dIn.readInt();
        }
        fIn.close();

        for (int z = 0; z < numRecs; z++)
        {
            System.out.println("offset[" + z + "] = " + offset[z]);
            raf.seek((int)offset[z]);

            System.out.println(raf.readUTF());
            System.out.println(raf.readInt());
            raf.readUTF();
            raf.readChar();
            System.out.println(raf.readInt());

        }

        raf.close();
    }
}


CreateDatabase.java
----------------------------
import java.io.*;

public class CreateDatabase
{
    private void init()
    {
        String[] name = new String[20];
        int[] license = new int[20];
        String[] address = new String[20];
        char[] gender = new char[20];
        int[] yearIssued = new int[20];
       
        for (int i = 0; i < 20; i++)
        {
            name[i] = "name " + i;
            license[i] = 1234560 + i;
            address[i] = "lives at " + i;
            gender[i] = 'M';
            yearIssued[i] = 1985;
        }

        try
        {
            RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
            DataOutputStream dos = new DataOutputStream(new FileOutputStream("license.ndx"));
            int startOfRecord = 0;
            for (int y = 0; y < 20; y++)
            {
                startOfRecord = (int)raf.getFilePointer();
                raf.writeUTF(name[y].concat("                               ").substring(0, 31));
                raf.writeInt(license[y]);
                raf.writeUTF(address[y].concat("                                    ").substring(0, 35));
                raf.writeChar(gender[y]);
                raf.writeInt(yearIssued[y]);
               
                dos.writeInt(license[y]);
                dos.writeInt(startOfRecord);
            }
           
            dos.close();
            raf.close();
        }
        catch (IOException ioe)
        {
            System.err.println("IOE: " + ioe.toString());
        }
    }

    public static void main(String[] args)
    {
        new CreateDatabase().init();
    }
}


AppendDatabase.java
------------------------------
import java.io.*;

public class AppendDatabase
{
    private void init()
    {
        String name = "Appended record";
        int license = 12345;
        String address = "Additional street";
        char gender = 'M';
        int yearIssued = 1980;
       
        try
        {
            RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
            DataOutputStream dos = new DataOutputStream(new FileOutputStream("license.ndx", true));

            raf.seek(raf.length());
            int startOfRecord = (int)raf.getFilePointer();
            raf.writeUTF(name.concat("                               ").substring(0, 31));
            raf.writeInt(license);
            raf.writeUTF(address.concat("                                    ").substring(0, 35));
            raf.writeChar(gender);
            raf.writeInt(yearIssued);
               
            dos.writeInt(license);
            dos.writeInt(startOfRecord);
           
            dos.close();
            raf.close();
        }
        catch (IOException ioe)
        {
            System.err.println("IOE: " + ioe.toString());
        }
    }

    public static void main(String[] args)
    {
        new AppendDatabase().init();
    }
}
0
 

Author Comment

by:bella2k4
Comment Utility
got it thanks
0
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
;-)
0
 

Author Comment

by:bella2k4
Comment Utility
im trying to make an index based upon 2 concatenated fields
firstname + " " + surname, but the database file regards them as 2 strings. Can you make the index based upon the concatenated fields but write the database file as 2 seperate fields without your offsets stuffing up. Ive done this and it seems to be possibly stuffing my offsets up.

name index file
------------------
Alison M Chappell 0
Barbara J Furza 480

displaying database according to name index file
------------------------------------------------
offset2[0] = 0
Alison M Chappell
696458
1963

offset2[2] = 80
Barbara J Furza
1504503
1962

-------------------------------------

See how the offsets for the 2nd record are wrong, its the same for several other records.
0
 
LVL 15

Expert Comment

by:jimmack
Comment Utility
I'm not sure, but it looks like you might be sorting the name array, but not swapping the corresponding elements in the offset array.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Viewers learn about the “while” loop and how to utilize it correctly in Java. Additionally, viewers begin exploring how to include conditional statements within a while loop and avoid an endless loop. Define While Loop: Basic Example: Explanatio…
This video teaches viewers about errors in exception handling.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now