Solved

Out of memory in massive files transmission

Posted on 2002-07-24
6
380 Views
Last Modified: 2011-08-18
I am transmiting lots of files through socket. The total number of the files could up to 1000, the total size up to 600M. At first the transmission is very fast, then it slow down. The memory in use keep on increase.

I use java command in form of

java -Xms32M -Xmx512M Myclass

I try to use garbage collection each time after send/receive one file.


      System.gc();      
      try {          
          Thread.sleep(1000);      
      }
      catch (Exception e){
      }


The speed slow down anyway.  Wether I put the sender and receiver on one Pc or on two, the result is the same.  I always get

java.lang.OutOfMemoryError
<<no stack trace available>>

Could someone give me some advice? Any suggestion is welcome and highly appreciated.


    public static boolean sendFile(String filename, ObjectOutputStream out) {

        Vector          fileVector = new Vector();

        System.out.println("send file " + filename);

        try {

            RandomAccessFile   tempFile = new RandomAccessFile(filename, "r");
            String line = "";

            while ((line = tempFile.readLine()) != null)
                fileVector.add(line);

            out.writeObject(fileVector);
            out.flush();
            tempFile.close();
        } catch (Exception e) {
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("send file " + filename + "  SUCCESS");
        return true;
    }


    public static boolean receiveFile(String filename, ObjectInputStream in) {

        Vector     fileVector = new Vector();

        System.out.println("receive file " + filename);

        try {

            int  i = filename.lastIndexOf("/");

            if (i > 0){
                File  temp = new File(filename.substring(0, i));
                temp.mkdirs();
            }

            fileVector = (Vector) in.readObject();

            RandomAccessFile  tempFile = new RandomAccessFile(filename, "rw");

            String            str;
            Iterator          iterator = fileVector.iterator();

            while(iterator.hasNext()) {

                str = (String)iterator.next();
                tempFile.writeBytes(str + "\n");
            }

            tempFile.close();

        } catch (Exception e) {
            System.out.println("error in receive and write file");
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("receive file " + filename + "  SUCCESS");
        return true;
    }
0
Comment
Question by:icefire
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 65 total points
ID: 7175546
Can you tell us which Java version you're running on and how much 'filename' varies?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 7175920
To begin with, give this a try:

public static boolean sendFile(String filename, ObjectOutputStream out) {
    File f = null;
    byte[] bytes = null;
    FileInputStream in = null;
    try {
      f = new File(filename);
      int size = (int)f.length();
      bytes = new byte[size];
      in = new FileInputStream(f);
      in.read(bytes,0,size);
      out.writeObject(bytes);
      out.flush();
      return true;
    }
    catch(Exception e){
      e.printStackTrace();
      bytes = null;
      return false;
    }
    finally {
      try {
        in.close();
        f = null;
        bytes = null;
        System.gc();
      }
      catch(IOException e){
        e.printStackTrace();
      }
    }
  }
0
 
LVL 92

Assisted Solution

by:objects
objects earned 65 total points
ID: 7175939
Calling gc() explicitly will not help you.
And all that sleep is going to do is slow you down.
Don't use RandomAccessFile and ObjectStreams's as they have lots of unnecessary overhead. Instead use BufferedInputStream and BufferedOutputStream's and read/writes bytes directly.
0
 
LVL 3

Assisted Solution

by:rjackman
rjackman earned 70 total points
ID: 7176353
agreed with objects and CEHJ moreover try using JDK 1.4
coz it got the improved java.io u can take the advantage of nio also
pls refer this
http://java.sun.com/j2se/1.4/docs/guide/nio/
cheers
RJ
0
 
LVL 35

Expert Comment

by:girionis
ID: 8951892
No comment has been added lately, so it's time to clean up this TA.

I will leave a recommendation in the Cleanup topic area that this question is:

- split points between CEHJ, objects and rjackman

Please leave any comments here within the
next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER !

girionis
Cleanup Volunteer
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How to parse custom JSON to POJO java 4 52
mockito example issue 8 62
jar file executable 12 38
javap not working 8 36
For customizing the look of your lightweight component and making it look lucid like it was made of glass. Or: how to make your component more Apple-ish ;) This tip assumes your component to be of rectangular shape and completely opaque. (COD…
This was posted to the Netbeans forum a Feb, 2010 and I also sent it to Verisign. Who didn't help much in my struggles to get my application signed. ------------------------- Start The idea here is to target your cell phones with the correct…
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
Viewers will learn about if statements in Java and their use The if statement: The condition required to create an if statement: Variations of if statements: An example using if statements:

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now