Solved

StringBuffer vs String : Serious Memory Waste Problems (also Memory leak)

Posted on 2002-05-02
5
1,938 Views
Last Modified: 2013-12-29
NOTE: You may jump directly to JUMP_POINT below
-----------------------------------------------

Nutshell : Should one use "return sb.toString()" or "return sb.substring(0, sb.length())"

My execution environment "Browser --> JSP --> MyClass --> JDBC --> Oracle"
JDK1.3.1, Tomcat3.2.3, Apache Web Server, Oracle OCI8/Thin drivers

I have to run queries that returns huge data (many pages). Once I get the data in the ResultSet object, I format that in the

HTML table format using the StringBuffer and then finally I convert the StringBuffer into String to pass it for the display

in the Broswer. Here is the code that I am using ...

public class MyClass {
    public String makeReport(String sql) {
         // Database Connection
         Connection con = ...
         Statement stmt = ...
         ResultSet rs = stmt.executeQuery(sql);
 
         StringBuffer sb = new StringBuffer();
         sb.append("<TABLE ALIGN=CENTER ...");

         try {
             while(rs.next()) {        <----- LOOPING
                 sb.append("<TR>");
                 for (int i = 1; i <= numColmns; i++) {    <----- LOOPING
                     sb.append("<TD>" +rs.getString(i) +"</TD>");
                 }
                 sb.append("</TR>");            
             }
         }
         catch (SQLException e) {
             ...
         }

         sb.append("</TABLE>");
         return sb.toString();  <----- ???
    }
}

I have used the same coding stuff in many other methods doing specific reporting that need specific formating (that I have

not shown here for simplicity). The report generated is offen huge. Here starts the problem...

JUMP_POINT:
-----------
(1). StringBuffer (sb) uses 32 bytes (16 chars) to startwith. The moment you append a single char after you added first 16 in it, it expands the internal char array to sb.capacity()*2 + 1 chars. That means if you add 17 chars to sb it will consume a memory of around 64B (ie a waste of almost 50% memory if you don't append any other chars). If you go further then for a large text you will have a situaion where just to store chars of size, say 1 MB, your StringBuffer might be consuming around 2 MB space. Certainly a big loss for me as my reports sometimes goes in the range of 10-15MB. So for a report of 10 MB if sb allocate around 20 MB it is a big problem. Isn't it? Any solution for this keeping the above code into consideration.

(2). But this is just a part of the problem. Since I have to return a String from my method I finally call sb.toString(). Now this method call will create a altogether NEW STRING INSTANCE i.e. it will require huge memory again. To make it worse (I read somewhere but not sure) that toString() will call "new String(StringBuffer)" to create the instance THAT allocates the
new String char array of the same size as that of the StringBuffer object i.e. it unnessasarily allocates the space where the String content is not there. Is it true? If yes then this call *may* further watse the memory to another 50% causing the total watse for me as 100%. Is there any shared memory concept among StringBuffer/String?

(3). If the above point is true then shall I call "sb.substring(0, sb.length())" ensuring that no extra space is wasted.

(4). Biggest problem though is the memory leak that I see in my Task Manager that shows the memory usage going up and up (sometimes it comes down also) causing the system sometimes hangs. Since the StringBuffer object in my above example has its scope within the method only, it should be GC'ed after the method returns (of-course whenever GC runs). However I doubt that it is not happening as there might be some reference kept in the new String created via sb.toString(). Anyone faced similar problem?

(5). Is there any known problem with the JSP PrintWriter object "out". After getting the report string I pass it to Browser using the "out.println(reportStr)".

(6). Any known issues with Tomcat3.2.3 specially in the area of Memory Leak in its Servlet Engine.

Please give your comments so that I can overcome this "memory waste" and "memory leak" problems.

Regards
0
Comment
Question by:javaq092999
  • 2
  • 2
5 Comments
 
LVL 16

Expert Comment

by:imladris
ID: 6986331
W.r.t. memory waste, you are quite right. StringBuffer is coded to expand by doubling its capacity. On the other hand, the conversion to String does not cause the use of extra memory. StringBuffer is coded to share its character data array with a String object. As long as the underlying StringBuffer does not change, no more memory will be used. Of course, as soon as you change the StringBuffer, the system must copy the StringBuffer to a String so as to avoid changing the String that toString produced.

If you can reasonably predict how much space you will need for the StringBuffer, you could manage all this by initializing the StringBuffer with the right amount of space (either in the constructor, or with setLength). If you can't tell beforehand, a reasonable solution might be to take the StringBuffer source and change it. Rename it to MyStringBuffer and change the expandCapacity algorithm to suit your purposes, and then use this new and improved object.

As for the potential memory leak and the hang, I'm not sure I can offer you anything specific there. Do you get an error message? Or does the system just hang?
0
 
LVL 92

Accepted Solution

by:
objects earned 100 total points
ID: 6987531
Simply don't build up a huge string to return to the client, and instead stream the data directly.

   public void makeReport(String sql, PrintWriter out) {
        // Database Connection
        Connection con = ...
        Statement stmt = ...
        ResultSet rs = stmt.executeQuery(sql);
 
        out.print("<TABLE ALIGN=CENTER ...");

        try {
            while(rs.next()) {        <----- LOOPING
                out.print("<TR>");
                for (int i = 1; i <= numColmns; i++) {    <----- LOOPING
                    out.print("<TD>" +rs.getString(i) +"</TD>");
                }
                out.print("</TR>");            
            }
        }
        catch (SQLException e) {
            ...
        }

        out.print("</TABLE>");
   }
}
0
 

Author Comment

by:javaq092999
ID: 6989523
After studying the source code for String and StringBuffer classes I am clear as how these APIs work specially on the memory front. Your (Object's) comment sounds great, however often I need to add something at the top in my string (ie using sb.insert(0, stuff)). The "stuff" is known only I process the data from the ResultSet. Since I will be streaming the data directly (*without caching*) I will not be able to do that. Anyway I have to cope with that :) Will put the comment once I implement this idea. Thanks to you both!
0
 

Author Comment

by:javaq092999
ID: 7001184
Well this problem helped me in understanding the String and StringBuffer classes more closely and learned that how easily the StringBuffer could lead to "Memory Waste". I finally changed my most of the metods to stream the text to the JspWriter directly as I was using the same in many other places. Thanks all and specially "object".
0
 
LVL 92

Expert Comment

by:objects
ID: 7002568
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
network + 7 73
Starting to learn JAVA, 7 47
JAVA part two 5 40
sites similar to codingbat to improve coding hanson skills 3 22
An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
Introduction This article is the second of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers the basic installation and configuration of the test automation tools used by…
Viewers learn about the “while” loop and how to utilize it correctly in Java. Additionally, viewers begin exploring how to include conditional statements within a while loop and avoid an endless loop. Define While Loop: Basic Example: Explanatio…
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now