Solved

Out of memory in massive files transmission

Posted on 2002-07-24
6
384 Views
Last Modified: 2011-08-18
I am transmiting lots of files through socket. The total number of the files could up to 1000, the total size up to 600M. At first the transmission is very fast, then it slow down. The memory in use keep on increase.

I use java command in form of

java -Xms32M -Xmx512M Myclass

I try to use garbage collection each time after send/receive one file.


      System.gc();      
      try {          
          Thread.sleep(1000);      
      }
      catch (Exception e){
      }


The speed slow down anyway.  Wether I put the sender and receiver on one Pc or on two, the result is the same.  I always get

java.lang.OutOfMemoryError
<<no stack trace available>>

Could someone give me some advice? Any suggestion is welcome and highly appreciated.


    public static boolean sendFile(String filename, ObjectOutputStream out) {

        Vector          fileVector = new Vector();

        System.out.println("send file " + filename);

        try {

            RandomAccessFile   tempFile = new RandomAccessFile(filename, "r");
            String line = "";

            while ((line = tempFile.readLine()) != null)
                fileVector.add(line);

            out.writeObject(fileVector);
            out.flush();
            tempFile.close();
        } catch (Exception e) {
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("send file " + filename + "  SUCCESS");
        return true;
    }


    public static boolean receiveFile(String filename, ObjectInputStream in) {

        Vector     fileVector = new Vector();

        System.out.println("receive file " + filename);

        try {

            int  i = filename.lastIndexOf("/");

            if (i > 0){
                File  temp = new File(filename.substring(0, i));
                temp.mkdirs();
            }

            fileVector = (Vector) in.readObject();

            RandomAccessFile  tempFile = new RandomAccessFile(filename, "rw");

            String            str;
            Iterator          iterator = fileVector.iterator();

            while(iterator.hasNext()) {

                str = (String)iterator.next();
                tempFile.writeBytes(str + "\n");
            }

            tempFile.close();

        } catch (Exception e) {
            System.out.println("error in receive and write file");
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("receive file " + filename + "  SUCCESS");
        return true;
    }
0
Comment
Question by:icefire
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 65 total points
ID: 7175546
Can you tell us which Java version you're running on and how much 'filename' varies?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 7175920
To begin with, give this a try:

public static boolean sendFile(String filename, ObjectOutputStream out) {
    File f = null;
    byte[] bytes = null;
    FileInputStream in = null;
    try {
      f = new File(filename);
      int size = (int)f.length();
      bytes = new byte[size];
      in = new FileInputStream(f);
      in.read(bytes,0,size);
      out.writeObject(bytes);
      out.flush();
      return true;
    }
    catch(Exception e){
      e.printStackTrace();
      bytes = null;
      return false;
    }
    finally {
      try {
        in.close();
        f = null;
        bytes = null;
        System.gc();
      }
      catch(IOException e){
        e.printStackTrace();
      }
    }
  }
0
 
LVL 92

Assisted Solution

by:objects
objects earned 65 total points
ID: 7175939
Calling gc() explicitly will not help you.
And all that sleep is going to do is slow you down.
Don't use RandomAccessFile and ObjectStreams's as they have lots of unnecessary overhead. Instead use BufferedInputStream and BufferedOutputStream's and read/writes bytes directly.
0
 
LVL 3

Assisted Solution

by:rjackman
rjackman earned 70 total points
ID: 7176353
agreed with objects and CEHJ moreover try using JDK 1.4
coz it got the improved java.io u can take the advantage of nio also
pls refer this
http://java.sun.com/j2se/1.4/docs/guide/nio/
cheers
RJ
0
 
LVL 35

Expert Comment

by:girionis
ID: 8951892
No comment has been added lately, so it's time to clean up this TA.

I will leave a recommendation in the Cleanup topic area that this question is:

- split points between CEHJ, objects and rjackman

Please leave any comments here within the
next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER !

girionis
Cleanup Volunteer
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
tomcat startup error 5 136
What's wrong with this code? 4 34
java.sql.SQLException: Access denied for user 'root'@'localhost' (using password: NO) 1 59
swing controls 2 21
Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question