using an index file to print out specific records

Hi im trying to print out some records from a raf file firstly in the order they are in, in the file then in order that they are according to the index file and Im pretty sure I have the right code. IT prints them out for the first part fine but when it comes to print them out according to the index file it says

Exception in thread "main" java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
at java.io.DataInputStream.readUTF(DataInputStream.java:580)
at java.io.RandomAccessFile.readUTF(RandomAccessFile.java:857)
at ShowDatabase.main(ShowDatabase.java:50)
line 50 is System.out.println(raf.readUTF()); down the bottom


public class ShowDatabase
{
static final int recLength = 113;

public static void main (String [] args ) throws Exception
{


RandomAccessFile raf = new RandomAccessFile("drivers.dbf","rw");
int numRecs = (int) raf.length()/recLength;

for (int x = 0; x < numRecs; x++)
{
System.out.println(raf.readUTF());
System.out.println(raf.readInt());
raf.readUTF();
raf.readChar();
System.out.println(raf.readInt());
}


File f = new File("license.ndx");
FileInputStream fIn = new FileInputStream(f);
DataInputStream dIn = new DataInputStream(fIn);

int offset[] = new int[numRecs];
int license[] = new int[numRecs];


for (int y = 0; y < numRecs; y++)
{
license[y] = dIn.readInt();
offset[y] = dIn.readInt();
}
fIn.close();

for(int z=0; z<numRecs; z++)
{

raf.seek((int)offset[z]);

System.out.println(raf.readUTF());
System.out.println(raf.readInt());
raf.readUTF();
raf.readChar();
System.out.println(raf.readInt());

}

raf.close();

}
}
bella2k4Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

jimmackCommented:
Add the following line before the "raf.see((int)offset[z]);"

System.out.println("offset[" + z + "] = " + offset[z]);

This will tell you the offset that it thinks record z is at.

It looks like you are trying to read beyond the end of the file.  You probably need to check your indices to make sure you are "seek"ing to the location you really want.

Also, if you are only reading the file, change the "rw" in the RandomAccessFile constructor to "r" for safety ;-)
0
bella2k4Author Commented:
offset[0] = 162

and the license index file contains
license number - index
------------------------------

696458  162
1504503  243
1519980  567
3501227  486
3510402  81
3513671  0
3513986  405
3516037  324
0
bella2k4Author Commented:
I figured out the problem its something to do with me trying to create an index based on firstname + " " + surname it doesnt like.

originally i was doing it like

         surname[x] = seqfile.readUTF().trim();
          firstname[x] = seqfile.readUTF().trim();
          name[x] = firstname[x] + " " + surname[x];      


and then


//writing the random access file
      for(int y=0; y<numRecs;++y)
      {
          raf.writeUTF(name[y].concat("                               ").substring(0,31) );

//31 being firstname 15 + surname 15 + 1 for spacing ?

          raf.writeInt(license[y]);
          raf.writeUTF(address[y].concat("                                    ").substring(0,35) );
          raf.writeChar(gender[y]);
          raf.writeInt(yearIssued[y]);
      }

Can anyone see anything wrong
0
Cloud Class® Course: Certified Penetration Testing

This CPTE Certified Penetration Testing Engineer course covers everything you need to know about becoming a Certified Penetration Testing Engineer. Career Path: Professional roles include Ethical Hackers, Security Consultants, System Administrators, and Chief Security Officers.

jimmackCommented:
I've built the system (I had to create some code to write the files ;-))

What I have found is that the entries are 80 bytes long, so my indices are:

0, 80, 160, 240 etc...

In your ndx file, they are 1 greater.

This means that when you try to read the UTF string, it is starting one byte further down the stream than it should, so the length of the UTF string is being corrupted and it is trying to read too many bytes.

The problem is in the way you create your .ndx file.  Your offset is 1 too big.
0
jimmackCommented:
If you want to check, here is the code I used to create the drivers.db and license.ndx files with test data:

Your existing ShowDatabase class should read the results of this OK.

BE CAREFUL.  BACKUP ANY DATA YOU WANT TO KEEP FIRST ;-)

import java.io.*;

public class CreateDatabase
{
    private void init()
    {
        String[] name = new String[20];
        int[] license = new int[20];
        String[] address = new String[20];
        char[] gender = new char[20];
        int[] yearIssued = new int[20];
       
        for (int i = 0; i < 20; i++)
        {
            name[i] = "name " + i;
            license[i] = 1234560 + i;
            address[i] = "lives at " + i;
            gender[i] = 'M';
            yearIssued[i] = 1985;
        }

        try
        {
            RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
            DataOutputStream dos = new DataOutputStream(new FileOutputStream("license.ndx"));
            for (int y = 0; y < 20; ++y)
            {
                raf.writeUTF(name[y].concat("                               ").substring(0, 31));
                raf.writeInt(license[y]);
                raf.writeUTF(address[y].concat("                                    ").substring(0, 35));
                raf.writeChar(gender[y]);
                raf.writeInt(yearIssued[y]);
               
                dos.writeInt(license[y]);
                dos.writeInt((int)raf.getFilePointer());
            }
           
            dos.close();
            raf.close();
        }
        catch (IOException ioe)
        {
            System.err.println("IOE: " + ioe.toString());
        }
    }

    public static void main(String[] args)
    {
        new CreateDatabase().init();
    }
}
0
bella2k4Author Commented:
          surname        String  17 (2+15)
           firstname      String 17 (2+15)
           name[]         String + " " + String
           gender         char     2 (Unicode character)
           address        String  37 (2+35)
           yearIssued     int      4
           licenseNumber  Int      4
                       
       Total record length = 81 characters

So im trying to base an index not only on license number but name which is firstname and surname,  when you concatenate them is it 15 + 15 + 1 space (" ")

like for the name something like.

 firstname[i] = "firstname " + i;
 surname[i] = "surname " + i;
 fullname[i] = firstname[i] + " " + surname[i];

raf.writeUTF(fullname[y].concat("                               ").substring(0, 31));

0
jimmackCommented:
I'm not sure if you are confirming what I've done or whether you are asking another question.

You created the file with the following writes (quoted from your earlier post with line numbers added by me):

//writing the random access file
      for(int y=0; y<numRecs;++y)
      {
1)          raf.writeUTF(name[y].concat("                               ").substring(0,31) );

//31 being firstname 15 + surname 15 + 1 for spacing ?

2)          raf.writeInt(license[y]);
3)          raf.writeUTF(address[y].concat("                                    ").substring(0,35) );
4)          raf.writeChar(gender[y]);
5)          raf.writeInt(yearIssued[y]);


1) a 31 character UTF String = 33 bytes
2) an int = 4 bytes
3) a 35 character UTF String = 37 bytes
4) a char = 2 bytes
5) an int = 4 bytes

33 + 4 + 37 + 2 + 4 = 80.

It doesn't really matter how the name is put together, you still only write the first 31 characters (using substring).

The .ndx file is just pairs of ints accessed using a DataInputStream, so lengths of license number doesn't really matter.

I really think the key is the size of the offset that you are calculating when you create the .ndx file.  Is there any chance you could post the code for that?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
bella2k4Author Commented:
oh i see what your getting at, i had the record length as if i was using first name and surname in a non concatenated form, so it will be 80, my bad.

Ill test it out see what happens
0
bella2k4Author Commented:
You fixed it well done,

firstly when im displaying according to name index file and license index file they seem to be in the exact same order. could just be a coinsidence that they work out to be the same order.

Secondly, checkout the offset values when displaying according to name index.


//straight out of the raf file

Alison M Chappell
696458
1963

Barbara J Furza
1504503
1962

//displayed according to license index

offset[0] = 160
Glen K Lewis
1519980
1943

offset[1] = 240
Maureen E Dixon
3501227
1949


//sorted according to name index

offset2[0] = 1769172846
Glen K Lewis
1519980
1943

offset2[1] = 1751216240
Maureen E Dixon
3501227
1949

0
bella2k4Author Commented:
nevermind found the problem
0
bella2k4Author Commented:
now I got this problem again

when it goes to use the name index

--------------------------------------------------
offset2[0] = 1769172846
Exception in thread "main" java.io.EOFException
        at java.io.RandomAccessFile.readUnsignedShort(RandomAccessFile.java:640)

        at java.io.DataInputStream.readUTF(DataInputStream.java:574)
        at java.io.RandomAccessFile.readUTF(RandomAccessFile.java:857)
        at ShowDatabase.main(ShowDatabase.java:86)
0
bella2k4Author Commented:
fixed it, i was reading in name as a int
me.idiot = true
0
bella2k4Author Commented:
new error
ive tried to append a record to the end of the database file and then display it

getting this error
last record -
-----------------------------------

jon doe
3843212
1908

------------------------------------
Exception in thread "main" java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:397)
        at ShowDatabase.main(ShowDatabase.java:43)
0
jimmackCommented:
I got the same error, but on the other file :-)

In my case, it's the RandomAccessFile that's running off the end.

In your case, did you update the index file with the appropriate data as well as the db file?
0
jimmackCommented:
OK.  I've got it ;-)

I now have three files (and I've included the source for all of the below).  I can also confirm that these files are tested and work ;-)

I have made the following changes (some of which you will have already done, some of which might be relevant to your problem):

ShowDatabase.java
1) Changed recLength constant to 80

CreateDatabase.java
1) Added a variable called "startOfRecord" before the "for(int y" loop, to keep track of the real offset of the start of this record (previously, I was using the offset *after* the record had been written, ie. the start of the *next* record :-()
2) Use this variable when writing to the .ndx file in the same loop.
3) Changed the ++y in the loop to y++ to make sure that y is not incremented to 1 for the first record (which should be 0).

AppendDatabase.java  (new file)
The class does a "seek" to the end of the db file in order to write to it.  It also updates the .ndx file.  NOTE:  When opening the .ndx file for writing, the second parameter in the constructor is "true", meaning - append to file.
Again, as with changes 1) and 2) for CreateDatabase.java, the recordPointer is stored before the db file is updated, then this value is appended to the .ndx file.

I hope this helps ;-)

Jim.


ShowDatabase.java
---------------------------
import java.io.*;

public class ShowDatabase
{
    final static int recLength = 80;

    public static void main(String[] args) throws Exception
    {

        RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
        int numRecs = (int)raf.length() / recLength;

        for (int x = 0; x < numRecs; x++)
        {
            System.out.println(raf.readUTF());
            System.out.println(raf.readInt());
            raf.readUTF();
            raf.readChar();
            System.out.println(raf.readInt());
        }

        File f = new File("license.ndx");
        FileInputStream fIn = new FileInputStream(f);
        DataInputStream dIn = new DataInputStream(fIn);

        int offset[] = new int[numRecs];
        int license[] = new int[numRecs];

        for (int y = 0; y < numRecs; y++)
        {
            license[y] = dIn.readInt();
            offset[y] = dIn.readInt();
        }
        fIn.close();

        for (int z = 0; z < numRecs; z++)
        {
            System.out.println("offset[" + z + "] = " + offset[z]);
            raf.seek((int)offset[z]);

            System.out.println(raf.readUTF());
            System.out.println(raf.readInt());
            raf.readUTF();
            raf.readChar();
            System.out.println(raf.readInt());

        }

        raf.close();
    }
}


CreateDatabase.java
----------------------------
import java.io.*;

public class CreateDatabase
{
    private void init()
    {
        String[] name = new String[20];
        int[] license = new int[20];
        String[] address = new String[20];
        char[] gender = new char[20];
        int[] yearIssued = new int[20];
       
        for (int i = 0; i < 20; i++)
        {
            name[i] = "name " + i;
            license[i] = 1234560 + i;
            address[i] = "lives at " + i;
            gender[i] = 'M';
            yearIssued[i] = 1985;
        }

        try
        {
            RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
            DataOutputStream dos = new DataOutputStream(new FileOutputStream("license.ndx"));
            int startOfRecord = 0;
            for (int y = 0; y < 20; y++)
            {
                startOfRecord = (int)raf.getFilePointer();
                raf.writeUTF(name[y].concat("                               ").substring(0, 31));
                raf.writeInt(license[y]);
                raf.writeUTF(address[y].concat("                                    ").substring(0, 35));
                raf.writeChar(gender[y]);
                raf.writeInt(yearIssued[y]);
               
                dos.writeInt(license[y]);
                dos.writeInt(startOfRecord);
            }
           
            dos.close();
            raf.close();
        }
        catch (IOException ioe)
        {
            System.err.println("IOE: " + ioe.toString());
        }
    }

    public static void main(String[] args)
    {
        new CreateDatabase().init();
    }
}


AppendDatabase.java
------------------------------
import java.io.*;

public class AppendDatabase
{
    private void init()
    {
        String name = "Appended record";
        int license = 12345;
        String address = "Additional street";
        char gender = 'M';
        int yearIssued = 1980;
       
        try
        {
            RandomAccessFile raf = new RandomAccessFile("drivers.dbf", "rw");
            DataOutputStream dos = new DataOutputStream(new FileOutputStream("license.ndx", true));

            raf.seek(raf.length());
            int startOfRecord = (int)raf.getFilePointer();
            raf.writeUTF(name.concat("                               ").substring(0, 31));
            raf.writeInt(license);
            raf.writeUTF(address.concat("                                    ").substring(0, 35));
            raf.writeChar(gender);
            raf.writeInt(yearIssued);
               
            dos.writeInt(license);
            dos.writeInt(startOfRecord);
           
            dos.close();
            raf.close();
        }
        catch (IOException ioe)
        {
            System.err.println("IOE: " + ioe.toString());
        }
    }

    public static void main(String[] args)
    {
        new AppendDatabase().init();
    }
}
0
bella2k4Author Commented:
got it thanks
0
jimmackCommented:
;-)
0
bella2k4Author Commented:
im trying to make an index based upon 2 concatenated fields
firstname + " " + surname, but the database file regards them as 2 strings. Can you make the index based upon the concatenated fields but write the database file as 2 seperate fields without your offsets stuffing up. Ive done this and it seems to be possibly stuffing my offsets up.

name index file
------------------
Alison M Chappell 0
Barbara J Furza 480

displaying database according to name index file
------------------------------------------------
offset2[0] = 0
Alison M Chappell
696458
1963

offset2[2] = 80
Barbara J Furza
1504503
1962

-------------------------------------

See how the offsets for the 2nd record are wrong, its the same for several other records.
0
jimmackCommented:
I'm not sure, but it looks like you might be sorting the name array, but not swapping the corresponding elements in the offset array.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.