Solved

Out of memory in massive files transmission

Posted on 2002-07-24
6
383 Views
Last Modified: 2011-08-18
I am transmiting lots of files through socket. The total number of the files could up to 1000, the total size up to 600M. At first the transmission is very fast, then it slow down. The memory in use keep on increase.

I use java command in form of

java -Xms32M -Xmx512M Myclass

I try to use garbage collection each time after send/receive one file.


      System.gc();      
      try {          
          Thread.sleep(1000);      
      }
      catch (Exception e){
      }


The speed slow down anyway.  Wether I put the sender and receiver on one Pc or on two, the result is the same.  I always get

java.lang.OutOfMemoryError
<<no stack trace available>>

Could someone give me some advice? Any suggestion is welcome and highly appreciated.


    public static boolean sendFile(String filename, ObjectOutputStream out) {

        Vector          fileVector = new Vector();

        System.out.println("send file " + filename);

        try {

            RandomAccessFile   tempFile = new RandomAccessFile(filename, "r");
            String line = "";

            while ((line = tempFile.readLine()) != null)
                fileVector.add(line);

            out.writeObject(fileVector);
            out.flush();
            tempFile.close();
        } catch (Exception e) {
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("send file " + filename + "  SUCCESS");
        return true;
    }


    public static boolean receiveFile(String filename, ObjectInputStream in) {

        Vector     fileVector = new Vector();

        System.out.println("receive file " + filename);

        try {

            int  i = filename.lastIndexOf("/");

            if (i > 0){
                File  temp = new File(filename.substring(0, i));
                temp.mkdirs();
            }

            fileVector = (Vector) in.readObject();

            RandomAccessFile  tempFile = new RandomAccessFile(filename, "rw");

            String            str;
            Iterator          iterator = fileVector.iterator();

            while(iterator.hasNext()) {

                str = (String)iterator.next();
                tempFile.writeBytes(str + "\n");
            }

            tempFile.close();

        } catch (Exception e) {
            System.out.println("error in receive and write file");
            e.printStackTrace();
            fileVector = null;
            return false;
        }

        fileVector = null;
        System.out.println("receive file " + filename + "  SUCCESS");
        return true;
    }
0
Comment
Question by:icefire
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 65 total points
ID: 7175546
Can you tell us which Java version you're running on and how much 'filename' varies?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 7175920
To begin with, give this a try:

public static boolean sendFile(String filename, ObjectOutputStream out) {
    File f = null;
    byte[] bytes = null;
    FileInputStream in = null;
    try {
      f = new File(filename);
      int size = (int)f.length();
      bytes = new byte[size];
      in = new FileInputStream(f);
      in.read(bytes,0,size);
      out.writeObject(bytes);
      out.flush();
      return true;
    }
    catch(Exception e){
      e.printStackTrace();
      bytes = null;
      return false;
    }
    finally {
      try {
        in.close();
        f = null;
        bytes = null;
        System.gc();
      }
      catch(IOException e){
        e.printStackTrace();
      }
    }
  }
0
 
LVL 92

Assisted Solution

by:objects
objects earned 65 total points
ID: 7175939
Calling gc() explicitly will not help you.
And all that sleep is going to do is slow you down.
Don't use RandomAccessFile and ObjectStreams's as they have lots of unnecessary overhead. Instead use BufferedInputStream and BufferedOutputStream's and read/writes bytes directly.
0
 
LVL 3

Assisted Solution

by:rjackman
rjackman earned 70 total points
ID: 7176353
agreed with objects and CEHJ moreover try using JDK 1.4
coz it got the improved java.io u can take the advantage of nio also
pls refer this
http://java.sun.com/j2se/1.4/docs/guide/nio/
cheers
RJ
0
 
LVL 35

Expert Comment

by:girionis
ID: 8951892
No comment has been added lately, so it's time to clean up this TA.

I will leave a recommendation in the Cleanup topic area that this question is:

- split points between CEHJ, objects and rjackman

Please leave any comments here within the
next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER !

girionis
Cleanup Volunteer
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
map related example 6 55
javap not working 8 58
jdbc error in jsp application 20 63
null output 3 35
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question