Solved

Out of memory in massive files transmission

Posted on 2002-07-24
6
386 Views
Last Modified: 2011-08-18
I am transmiting lots of files through socket. The total number of the files could up to 1000, the total size up to 600M. At first the transmission is very fast, then it slow down. The memory in use keep on increase.

I use java command in form of

java -Xms32M -Xmx512M Myclass

I try to use garbage collection each time after send/receive one file.


      System.gc();      
      try {          
          Thread.sleep(1000);      
      }
      catch (Exception e){
      }


The speed slow down anyway.  Wether I put the sender and receiver on one Pc or on two, the result is the same.  I always get

java.lang.OutOfMemoryError
<<no stack trace available>>

Could someone give me some advice? Any suggestion is welcome and highly appreciated.


    public static boolean sendFile(String filename, ObjectOutputStream out) {

        Vector          fileVector = new Vector();

        System.out.println("send file " + filename);

        try {

            RandomAccessFile   tempFile = new RandomAccessFile(filename, "r");
            String line = "";

            while ((line = tempFile.readLine()) != null)
                fileVector.add(line);

            out.writeObject(fileVector);
            out.flush();
            tempFile.close();
        } catch (Exception e) {
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("send file " + filename + "  SUCCESS");
        return true;
    }


    public static boolean receiveFile(String filename, ObjectInputStream in) {

        Vector     fileVector = new Vector();

        System.out.println("receive file " + filename);

        try {

            int  i = filename.lastIndexOf("/");

            if (i > 0){
                File  temp = new File(filename.substring(0, i));
                temp.mkdirs();
            }

            fileVector = (Vector) in.readObject();

            RandomAccessFile  tempFile = new RandomAccessFile(filename, "rw");

            String            str;
            Iterator          iterator = fileVector.iterator();

            while(iterator.hasNext()) {

                str = (String)iterator.next();
                tempFile.writeBytes(str + "\n");
            }

            tempFile.close();

        } catch (Exception e) {
            System.out.println("error in receive and write file");
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("receive file " + filename + "  SUCCESS");
        return true;
    }
0
Comment
Question by:icefire
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 65 total points
ID: 7175546
Can you tell us which Java version you're running on and how much 'filename' varies?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 7175920
To begin with, give this a try:

public static boolean sendFile(String filename, ObjectOutputStream out) {
    File f = null;
    byte[] bytes = null;
    FileInputStream in = null;
    try {
      f = new File(filename);
      int size = (int)f.length();
      bytes = new byte[size];
      in = new FileInputStream(f);
      in.read(bytes,0,size);
      out.writeObject(bytes);
      out.flush();
      return true;
    }
    catch(Exception e){
      e.printStackTrace();
      bytes = null;
      return false;
    }
    finally {
      try {
        in.close();
        f = null;
        bytes = null;
        System.gc();
      }
      catch(IOException e){
        e.printStackTrace();
      }
    }
  }
0
 
LVL 92

Assisted Solution

by:objects
objects earned 65 total points
ID: 7175939
Calling gc() explicitly will not help you.
And all that sleep is going to do is slow you down.
Don't use RandomAccessFile and ObjectStreams's as they have lots of unnecessary overhead. Instead use BufferedInputStream and BufferedOutputStream's and read/writes bytes directly.
0
 
LVL 3

Assisted Solution

by:rjackman
rjackman earned 70 total points
ID: 7176353
agreed with objects and CEHJ moreover try using JDK 1.4
coz it got the improved java.io u can take the advantage of nio also
pls refer this
http://java.sun.com/j2se/1.4/docs/guide/nio/
cheers
RJ
0
 
LVL 35

Expert Comment

by:girionis
ID: 8951892
No comment has been added lately, so it's time to clean up this TA.

I will leave a recommendation in the Cleanup topic area that this question is:

- split points between CEHJ, objects and rjackman

Please leave any comments here within the
next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER !

girionis
Cleanup Volunteer
0

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

After being asked a question last year, I went into one of my moods where I did some research and code just for the fun and learning of it all.  Subsequently, from this journey, I put together this article on "Range Searching Using Visual Basic.NET …
Introduction This article is the first of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article explains our test automation goals. Then rationale is given for the tools we use to a…
This video teaches viewers about errors in exception handling.
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …
Suggested Courses

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question