Solved

StringBuffer vs String : Serious Memory Waste Problems (also Memory leak)

Posted on 2002-05-02
5
1,951 Views
Last Modified: 2013-12-29
NOTE: You may jump directly to JUMP_POINT below
-----------------------------------------------

Nutshell : Should one use "return sb.toString()" or "return sb.substring(0, sb.length())"

My execution environment "Browser --> JSP --> MyClass --> JDBC --> Oracle"
JDK1.3.1, Tomcat3.2.3, Apache Web Server, Oracle OCI8/Thin drivers

I have to run queries that returns huge data (many pages). Once I get the data in the ResultSet object, I format that in the

HTML table format using the StringBuffer and then finally I convert the StringBuffer into String to pass it for the display

in the Broswer. Here is the code that I am using ...

public class MyClass {
    public String makeReport(String sql) {
         // Database Connection
         Connection con = ...
         Statement stmt = ...
         ResultSet rs = stmt.executeQuery(sql);
 
         StringBuffer sb = new StringBuffer();
         sb.append("<TABLE ALIGN=CENTER ...");

         try {
             while(rs.next()) {        <----- LOOPING
                 sb.append("<TR>");
                 for (int i = 1; i <= numColmns; i++) {    <----- LOOPING
                     sb.append("<TD>" +rs.getString(i) +"</TD>");
                 }
                 sb.append("</TR>");            
             }
         }
         catch (SQLException e) {
             ...
         }

         sb.append("</TABLE>");
         return sb.toString();  <----- ???
    }
}

I have used the same coding stuff in many other methods doing specific reporting that need specific formating (that I have

not shown here for simplicity). The report generated is offen huge. Here starts the problem...

JUMP_POINT:
-----------
(1). StringBuffer (sb) uses 32 bytes (16 chars) to startwith. The moment you append a single char after you added first 16 in it, it expands the internal char array to sb.capacity()*2 + 1 chars. That means if you add 17 chars to sb it will consume a memory of around 64B (ie a waste of almost 50% memory if you don't append any other chars). If you go further then for a large text you will have a situaion where just to store chars of size, say 1 MB, your StringBuffer might be consuming around 2 MB space. Certainly a big loss for me as my reports sometimes goes in the range of 10-15MB. So for a report of 10 MB if sb allocate around 20 MB it is a big problem. Isn't it? Any solution for this keeping the above code into consideration.

(2). But this is just a part of the problem. Since I have to return a String from my method I finally call sb.toString(). Now this method call will create a altogether NEW STRING INSTANCE i.e. it will require huge memory again. To make it worse (I read somewhere but not sure) that toString() will call "new String(StringBuffer)" to create the instance THAT allocates the
new String char array of the same size as that of the StringBuffer object i.e. it unnessasarily allocates the space where the String content is not there. Is it true? If yes then this call *may* further watse the memory to another 50% causing the total watse for me as 100%. Is there any shared memory concept among StringBuffer/String?

(3). If the above point is true then shall I call "sb.substring(0, sb.length())" ensuring that no extra space is wasted.

(4). Biggest problem though is the memory leak that I see in my Task Manager that shows the memory usage going up and up (sometimes it comes down also) causing the system sometimes hangs. Since the StringBuffer object in my above example has its scope within the method only, it should be GC'ed after the method returns (of-course whenever GC runs). However I doubt that it is not happening as there might be some reference kept in the new String created via sb.toString(). Anyone faced similar problem?

(5). Is there any known problem with the JSP PrintWriter object "out". After getting the report string I pass it to Browser using the "out.println(reportStr)".

(6). Any known issues with Tomcat3.2.3 specially in the area of Memory Leak in its Servlet Engine.

Please give your comments so that I can overcome this "memory waste" and "memory leak" problems.

Regards
0
Comment
Question by:javaq092999
  • 2
  • 2
5 Comments
 
LVL 16

Expert Comment

by:imladris
ID: 6986331
W.r.t. memory waste, you are quite right. StringBuffer is coded to expand by doubling its capacity. On the other hand, the conversion to String does not cause the use of extra memory. StringBuffer is coded to share its character data array with a String object. As long as the underlying StringBuffer does not change, no more memory will be used. Of course, as soon as you change the StringBuffer, the system must copy the StringBuffer to a String so as to avoid changing the String that toString produced.

If you can reasonably predict how much space you will need for the StringBuffer, you could manage all this by initializing the StringBuffer with the right amount of space (either in the constructor, or with setLength). If you can't tell beforehand, a reasonable solution might be to take the StringBuffer source and change it. Rename it to MyStringBuffer and change the expandCapacity algorithm to suit your purposes, and then use this new and improved object.

As for the potential memory leak and the hang, I'm not sure I can offer you anything specific there. Do you get an error message? Or does the system just hang?
0
 
LVL 92

Accepted Solution

by:
objects earned 100 total points
ID: 6987531
Simply don't build up a huge string to return to the client, and instead stream the data directly.

   public void makeReport(String sql, PrintWriter out) {
        // Database Connection
        Connection con = ...
        Statement stmt = ...
        ResultSet rs = stmt.executeQuery(sql);
 
        out.print("<TABLE ALIGN=CENTER ...");

        try {
            while(rs.next()) {        <----- LOOPING
                out.print("<TR>");
                for (int i = 1; i <= numColmns; i++) {    <----- LOOPING
                    out.print("<TD>" +rs.getString(i) +"</TD>");
                }
                out.print("</TR>");            
            }
        }
        catch (SQLException e) {
            ...
        }

        out.print("</TABLE>");
   }
}
0
 

Author Comment

by:javaq092999
ID: 6989523
After studying the source code for String and StringBuffer classes I am clear as how these APIs work specially on the memory front. Your (Object's) comment sounds great, however often I need to add something at the top in my string (ie using sb.insert(0, stuff)). The "stuff" is known only I process the data from the ResultSet. Since I will be streaming the data directly (*without caching*) I will not be able to do that. Anyway I have to cope with that :) Will put the comment once I implement this idea. Thanks to you both!
0
 

Author Comment

by:javaq092999
ID: 7001184
Well this problem helped me in understanding the String and StringBuffer classes more closely and learned that how easily the StringBuffer could lead to "Memory Waste". I finally changed my most of the metods to stream the text to the JspWriter directly as I was using the same in many other places. Thanks all and specially "object".
0
 
LVL 92

Expert Comment

by:objects
ID: 7002568
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
simple java question 3 59
mysql jsp example issue 32 48
ejb mdb examples 1 6
Notify sent to other threads in Java 9 33
For customizing the look of your lightweight component and making it look opaque like it was made of plastic.  This tip assumes your component to be of rectangular shape and completely opaque.   (CODE)
For beginner Java programmers or at least those new to the Eclipse IDE, the following tutorial will show some (four) ways in which you can import your Java projects to your Eclipse workbench. Introduction While learning Java can be done with…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.

790 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question