Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Out of memory in massive files transmission

Posted on 2002-07-24
6
Medium Priority
?
388 Views
Last Modified: 2011-08-18
I am transmiting lots of files through socket. The total number of the files could up to 1000, the total size up to 600M. At first the transmission is very fast, then it slow down. The memory in use keep on increase.

I use java command in form of

java -Xms32M -Xmx512M Myclass

I try to use garbage collection each time after send/receive one file.


      System.gc();      
      try {          
          Thread.sleep(1000);      
      }
      catch (Exception e){
      }


The speed slow down anyway.  Wether I put the sender and receiver on one Pc or on two, the result is the same.  I always get

java.lang.OutOfMemoryError
<<no stack trace available>>

Could someone give me some advice? Any suggestion is welcome and highly appreciated.


    public static boolean sendFile(String filename, ObjectOutputStream out) {

        Vector          fileVector = new Vector();

        System.out.println("send file " + filename);

        try {

            RandomAccessFile   tempFile = new RandomAccessFile(filename, "r");
            String line = "";

            while ((line = tempFile.readLine()) != null)
                fileVector.add(line);

            out.writeObject(fileVector);
            out.flush();
            tempFile.close();
        } catch (Exception e) {
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("send file " + filename + "  SUCCESS");
        return true;
    }


    public static boolean receiveFile(String filename, ObjectInputStream in) {

        Vector     fileVector = new Vector();

        System.out.println("receive file " + filename);

        try {

            int  i = filename.lastIndexOf("/");

            if (i > 0){
                File  temp = new File(filename.substring(0, i));
                temp.mkdirs();
            }

            fileVector = (Vector) in.readObject();

            RandomAccessFile  tempFile = new RandomAccessFile(filename, "rw");

            String            str;
            Iterator          iterator = fileVector.iterator();

            while(iterator.hasNext()) {

                str = (String)iterator.next();
                tempFile.writeBytes(str + "\n");
            }

            tempFile.close();

        } catch (Exception e) {
            System.out.println("error in receive and write file");
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("receive file " + filename + "  SUCCESS");
        return true;
    }
0
Comment
Question by:icefire
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 260 total points
ID: 7175546
Can you tell us which Java version you're running on and how much 'filename' varies?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 7175920
To begin with, give this a try:

public static boolean sendFile(String filename, ObjectOutputStream out) {
    File f = null;
    byte[] bytes = null;
    FileInputStream in = null;
    try {
      f = new File(filename);
      int size = (int)f.length();
      bytes = new byte[size];
      in = new FileInputStream(f);
      in.read(bytes,0,size);
      out.writeObject(bytes);
      out.flush();
      return true;
    }
    catch(Exception e){
      e.printStackTrace();
      bytes = null;
      return false;
    }
    finally {
      try {
        in.close();
        f = null;
        bytes = null;
        System.gc();
      }
      catch(IOException e){
        e.printStackTrace();
      }
    }
  }
0
 
LVL 92

Assisted Solution

by:objects
objects earned 260 total points
ID: 7175939
Calling gc() explicitly will not help you.
And all that sleep is going to do is slow you down.
Don't use RandomAccessFile and ObjectStreams's as they have lots of unnecessary overhead. Instead use BufferedInputStream and BufferedOutputStream's and read/writes bytes directly.
0
 
LVL 3

Assisted Solution

by:rjackman
rjackman earned 280 total points
ID: 7176353
agreed with objects and CEHJ moreover try using JDK 1.4
coz it got the improved java.io u can take the advantage of nio also
pls refer this
http://java.sun.com/j2se/1.4/docs/guide/nio/
cheers
RJ
0
 
LVL 35

Expert Comment

by:girionis
ID: 8951892
No comment has been added lately, so it's time to clean up this TA.

I will leave a recommendation in the Cleanup topic area that this question is:

- split points between CEHJ, objects and rjackman

Please leave any comments here within the
next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER !

girionis
Cleanup Volunteer
0

Featured Post

The top UI technologies you need to be aware of

An important part of the job as a front-end developer is to stay up to date and in contact with new tools, trends and workflows. That’s why you cannot miss this upcoming webinar to explore the latest trends in UI technologies!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…
Suggested Courses

715 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question