Link to home
Start Free TrialLog in
Avatar of robthewolf
robthewolfFlag for Israel

asked on

Memory Leak in Java

I have code similar to the code below.  It waits for an object to arrive on the object input stream that it receives over a TCP connection.
When an object of type DataMessage is received it will then put it on a queue to be handled by a thread that is started earlier in the startThread() method.  (getX and getY return strings and getZ returns and object)
ois is of type ObjectInputStream
the queue is declared as:
        this.queue = new ArrayBlockingQueue<DataMessage>(10000);
 
I am aware that this line: final DataMessage msg = resp;
seems unnecessary, it is! It is legacy code and should be deleted.  However I was wondering if that was what was leading to my memory leak, particularly the use of final in this context.
My profiler is telling me that my ObjectInputStream is taking a huge amount of memory.  The when I descend through the tree is shows ObjectInputStream$HandleTable, and below that java.lang.Object[] holding the largest amount of memeory.

What I would like to know is whether the final is potentially causing this problem or it is something else to do with how I have written my code.  Or it is not obvious from here.

Thanks for your time.
public boolean waitForServerResponse() throws SocketException, IOException, ClassNotFoundException {

        DataMessage resp = null;
        try{
            resp = (DataMessage) ois.readObject();
        } catch (EOFException e) {
            log.error("EOFException when receiving data - ignoring");
        }
        final DataMessage msg = resp;
        queue.offer(msg);
 
        return false;
    }

    private void startQueue() {
        
        new Thread(new Runnable() {
            public void run() {
                while(isReady()) {
                        try {
                            DataMessage msg = queue.take();
                            callback.newMsg(msg.getX(),msg.getY(),  msg.getZ());
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                }
            }
        }).start();
    }

Open in new window

Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

The problem is most likely going to be the size of the object itself
I can't think of final causing issues here, btw what is that object array holds?
Avatar of robthewolf

ASKER

Can you explain what you mean.  Over time I have more and more objects of this type and the size continues to grow.  Admittedly the size of the object will mater as a smaller object will allow more objects to be created before it runs out of memory.  As far as I can tell the queue is not full, not even close, so why am I holding so many objects in memory and what is causing this.
ksivananth
I presume the object array is something to do with the internal workings of ObjectInputStream.  Presumable it would be a list of my DataMessage objects but as I am pretty new to the profiler I am finding it very hard to interpret.  I am using the YourKit java profiler.
the size of the Object array is about 360000.  It is not possible that it has that many objects waiting to be read.  So I don't get why the ObjectInputStream is holding on to these objects once they have been read.
ASKER CERTIFIED SOLUTION
Avatar of anilallewar
anilallewar
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Before you run out of memory, do an experiment: write the object to a ByteArrayOutputStream and find the size of the resulting byte array. That will give you an idea of what you're dealing with
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
No the ObjectInputStream does not close.  I guess it would lose the references if it did.  I have the client server connection open.  Once it opens it assigns the ObjectInputStream from the connection to ois and then waits for incoming messages.  It stays open indefinitely.
Is this not the correct way to do things?
Let me just give you a run down of the client server interaction.
Server run waiting for client connection.
Client runs initializing connection to server, and opening the ObjectInputStream.
Server registers new client.
Client sends request.
Server sends data continuously until client closes.
Client receives and handles data. (code above)
>> I guess it would lose the references if it did.

No - you should get the object(s) out of it and close it pronto. The references are then stored elsewhere (the queue)
>> No - you should get the object(s) out of it and close it pronto. The references are then stored elsewhere (the queue)

I dont think that I mean the references.  Well kind of in the sense that if they ObjectInputStream is closed then will the memory be freed?

If I close ObjectInputStream then I will get an exception on the server when it tries to send the next object.
>>If I close ObjectInputStream then I will get an exception on the server when it tries to send the next object.

You need to arrange things such that they communicate in 'bursts', closing and opening sockets as necessary (although of course there would be a permanently listening ServerSocket). The consequence of leaving channels open is that your stream will continually pile up its buffers until you run out of memory
I work with financial data, specifically forex, the data comes in 24 hours a day 5 days a week and it arrives as frequently as a few times a second.  Can you suggest a method/ psuedo code of how I can handle this without leaving my connection open all the time please.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
It's not the 'correct solution' no, but it's possibly worth a try as a workaround
I am going to try it and see how it goes.
Do you regard the correct solution as closing and reopening the client/server connection?
>>Do you regard the correct solution as closing and reopening the client/server connection?

Yes. The only times when you shouldn't do that is when the overhead of establishing the connection is critical
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
> For optimal operation, your app should read the object and then close the socket/stream

closing the stream is *not* optimal :)