Memory Leak in Java

I have code similar to the code below.  It waits for an object to arrive on the object input stream that it receives over a TCP connection.
When an object of type DataMessage is received it will then put it on a queue to be handled by a thread that is started earlier in the startThread() method.  (getX and getY return strings and getZ returns and object)
ois is of type ObjectInputStream
the queue is declared as:
        this.queue = new ArrayBlockingQueue<DataMessage>(10000);
 
I am aware that this line: final DataMessage msg = resp;
seems unnecessary, it is! It is legacy code and should be deleted.  However I was wondering if that was what was leading to my memory leak, particularly the use of final in this context.
My profiler is telling me that my ObjectInputStream is taking a huge amount of memory.  The when I descend through the tree is shows ObjectInputStream$HandleTable, and below that java.lang.Object[] holding the largest amount of memeory.

What I would like to know is whether the final is potentially causing this problem or it is something else to do with how I have written my code.  Or it is not obvious from here.

Thanks for your time.
public boolean waitForServerResponse() throws SocketException, IOException, ClassNotFoundException {

        DataMessage resp = null;
        try{
            resp = (DataMessage) ois.readObject();
        } catch (EOFException e) {
            log.error("EOFException when receiving data - ignoring");
        }
        final DataMessage msg = resp;
        queue.offer(msg);
 
        return false;
    }

    private void startQueue() {
        
        new Thread(new Runnable() {
            public void run() {
                while(isReady()) {
                        try {
                            DataMessage msg = queue.take();
                            callback.newMsg(msg.getX(),msg.getY(),  msg.getZ());
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                }
            }
        }).start();
    }

Open in new window

LVL 8
robthewolfAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

CEHJCommented:
The problem is most likely going to be the size of the object itself
ksivananthCommented:
I can't think of final causing issues here, btw what is that object array holds?
robthewolfAuthor Commented:
Can you explain what you mean.  Over time I have more and more objects of this type and the size continues to grow.  Admittedly the size of the object will mater as a smaller object will allow more objects to be created before it runs out of memory.  As far as I can tell the queue is not full, not even close, so why am I holding so many objects in memory and what is causing this.
Fundamentals of JavaScript

Learn the fundamentals of the popular programming language JavaScript so that you can explore the realm of web development.

robthewolfAuthor Commented:
ksivananth
I presume the object array is something to do with the internal workings of ObjectInputStream.  Presumable it would be a list of my DataMessage objects but as I am pretty new to the profiler I am finding it very hard to interpret.  I am using the YourKit java profiler.
the size of the Object array is about 360000.  It is not possible that it has that many objects waiting to be read.  So I don't get why the ObjectInputStream is holding on to these objects once they have been read.
anilallewarCommented:
Is it possible that your ObjectInputStream reads objects and never closes? If that is the case, it might be that is continues to hold the other references.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CEHJCommented:
Before you run out of memory, do an experiment: write the object to a ByteArrayOutputStream and find the size of the resulting byte array. That will give you an idea of what you're dealing with
CEHJCommented:
http:#33946421 is a good point. For optimal operation, your app should read the object and then close the socket/stream
robthewolfAuthor Commented:
No the ObjectInputStream does not close.  I guess it would lose the references if it did.  I have the client server connection open.  Once it opens it assigns the ObjectInputStream from the connection to ois and then waits for incoming messages.  It stays open indefinitely.
Is this not the correct way to do things?
robthewolfAuthor Commented:
Let me just give you a run down of the client server interaction.
Server run waiting for client connection.
Client runs initializing connection to server, and opening the ObjectInputStream.
Server registers new client.
Client sends request.
Server sends data continuously until client closes.
Client receives and handles data. (code above)
CEHJCommented:
>> I guess it would lose the references if it did.

No - you should get the object(s) out of it and close it pronto. The references are then stored elsewhere (the queue)
robthewolfAuthor Commented:
>> No - you should get the object(s) out of it and close it pronto. The references are then stored elsewhere (the queue)

I dont think that I mean the references.  Well kind of in the sense that if they ObjectInputStream is closed then will the memory be freed?

If I close ObjectInputStream then I will get an exception on the server when it tries to send the next object.
CEHJCommented:
>>If I close ObjectInputStream then I will get an exception on the server when it tries to send the next object.

You need to arrange things such that they communicate in 'bursts', closing and opening sockets as necessary (although of course there would be a permanently listening ServerSocket). The consequence of leaving channels open is that your stream will continually pile up its buffers until you run out of memory
robthewolfAuthor Commented:
I work with financial data, specifically forex, the data comes in 24 hours a day 5 days a week and it arrives as frequently as a few times a second.  Can you suggest a method/ psuedo code of how I can handle this without leaving my connection open all the time please.
robthewolfAuthor Commented:
I just read about a method in ObjectOutputStream called reset().
If I understand it correctly it should reset BOTH streams thus solving my problem without having to disconnect and reconnect the client server connections.
Can you confirm that this is the correct solution
CEHJCommented:
It's not the 'correct solution' no, but it's possibly worth a try as a workaround
robthewolfAuthor Commented:
I am going to try it and see how it goes.
Do you regard the correct solution as closing and reopening the client/server connection?
CEHJCommented:
>>Do you regard the correct solution as closing and reopening the client/server connection?

Yes. The only times when you shouldn't do that is when the overhead of establishing the connection is critical
Mick BarryJava DeveloperCommented:
you should be calling reset(), otherwise it will hold onto the entire object graph of anything being streamed. You don't need to reconnect everytime just to achieve that.

reset() releases the references to the streamed object and should be called once there is no longer a need to hold onto them.
Mick BarryJava DeveloperCommented:
> For optimal operation, your app should read the object and then close the socket/stream

closing the stream is *not* optimal :)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.