Solved

StringBuffer vs String : Serious Memory Waste Problems (also Memory leak)

Posted on 2002-05-02
5
1,970 Views
Last Modified: 2013-12-29
NOTE: You may jump directly to JUMP_POINT below
-----------------------------------------------

Nutshell : Should one use "return sb.toString()" or "return sb.substring(0, sb.length())"

My execution environment "Browser --> JSP --> MyClass --> JDBC --> Oracle"
JDK1.3.1, Tomcat3.2.3, Apache Web Server, Oracle OCI8/Thin drivers

I have to run queries that returns huge data (many pages). Once I get the data in the ResultSet object, I format that in the

HTML table format using the StringBuffer and then finally I convert the StringBuffer into String to pass it for the display

in the Broswer. Here is the code that I am using ...

public class MyClass {
    public String makeReport(String sql) {
         // Database Connection
         Connection con = ...
         Statement stmt = ...
         ResultSet rs = stmt.executeQuery(sql);
 
         StringBuffer sb = new StringBuffer();
         sb.append("<TABLE ALIGN=CENTER ...");

         try {
             while(rs.next()) {        <----- LOOPING
                 sb.append("<TR>");
                 for (int i = 1; i <= numColmns; i++) {    <----- LOOPING
                     sb.append("<TD>" +rs.getString(i) +"</TD>");
                 }
                 sb.append("</TR>");            
             }
         }
         catch (SQLException e) {
             ...
         }

         sb.append("</TABLE>");
         return sb.toString();  <----- ???
    }
}

I have used the same coding stuff in many other methods doing specific reporting that need specific formating (that I have

not shown here for simplicity). The report generated is offen huge. Here starts the problem...

JUMP_POINT:
-----------
(1). StringBuffer (sb) uses 32 bytes (16 chars) to startwith. The moment you append a single char after you added first 16 in it, it expands the internal char array to sb.capacity()*2 + 1 chars. That means if you add 17 chars to sb it will consume a memory of around 64B (ie a waste of almost 50% memory if you don't append any other chars). If you go further then for a large text you will have a situaion where just to store chars of size, say 1 MB, your StringBuffer might be consuming around 2 MB space. Certainly a big loss for me as my reports sometimes goes in the range of 10-15MB. So for a report of 10 MB if sb allocate around 20 MB it is a big problem. Isn't it? Any solution for this keeping the above code into consideration.

(2). But this is just a part of the problem. Since I have to return a String from my method I finally call sb.toString(). Now this method call will create a altogether NEW STRING INSTANCE i.e. it will require huge memory again. To make it worse (I read somewhere but not sure) that toString() will call "new String(StringBuffer)" to create the instance THAT allocates the
new String char array of the same size as that of the StringBuffer object i.e. it unnessasarily allocates the space where the String content is not there. Is it true? If yes then this call *may* further watse the memory to another 50% causing the total watse for me as 100%. Is there any shared memory concept among StringBuffer/String?

(3). If the above point is true then shall I call "sb.substring(0, sb.length())" ensuring that no extra space is wasted.

(4). Biggest problem though is the memory leak that I see in my Task Manager that shows the memory usage going up and up (sometimes it comes down also) causing the system sometimes hangs. Since the StringBuffer object in my above example has its scope within the method only, it should be GC'ed after the method returns (of-course whenever GC runs). However I doubt that it is not happening as there might be some reference kept in the new String created via sb.toString(). Anyone faced similar problem?

(5). Is there any known problem with the JSP PrintWriter object "out". After getting the report string I pass it to Browser using the "out.println(reportStr)".

(6). Any known issues with Tomcat3.2.3 specially in the area of Memory Leak in its Servlet Engine.

Please give your comments so that I can overcome this "memory waste" and "memory leak" problems.

Regards
0
Comment
Question by:javaq092999
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
5 Comments
 
LVL 16

Expert Comment

by:imladris
ID: 6986331
W.r.t. memory waste, you are quite right. StringBuffer is coded to expand by doubling its capacity. On the other hand, the conversion to String does not cause the use of extra memory. StringBuffer is coded to share its character data array with a String object. As long as the underlying StringBuffer does not change, no more memory will be used. Of course, as soon as you change the StringBuffer, the system must copy the StringBuffer to a String so as to avoid changing the String that toString produced.

If you can reasonably predict how much space you will need for the StringBuffer, you could manage all this by initializing the StringBuffer with the right amount of space (either in the constructor, or with setLength). If you can't tell beforehand, a reasonable solution might be to take the StringBuffer source and change it. Rename it to MyStringBuffer and change the expandCapacity algorithm to suit your purposes, and then use this new and improved object.

As for the potential memory leak and the hang, I'm not sure I can offer you anything specific there. Do you get an error message? Or does the system just hang?
0
 
LVL 92

Accepted Solution

by:
objects earned 100 total points
ID: 6987531
Simply don't build up a huge string to return to the client, and instead stream the data directly.

   public void makeReport(String sql, PrintWriter out) {
        // Database Connection
        Connection con = ...
        Statement stmt = ...
        ResultSet rs = stmt.executeQuery(sql);
 
        out.print("<TABLE ALIGN=CENTER ...");

        try {
            while(rs.next()) {        <----- LOOPING
                out.print("<TR>");
                for (int i = 1; i <= numColmns; i++) {    <----- LOOPING
                    out.print("<TD>" +rs.getString(i) +"</TD>");
                }
                out.print("</TR>");            
            }
        }
        catch (SQLException e) {
            ...
        }

        out.print("</TABLE>");
   }
}
0
 

Author Comment

by:javaq092999
ID: 6989523
After studying the source code for String and StringBuffer classes I am clear as how these APIs work specially on the memory front. Your (Object's) comment sounds great, however often I need to add something at the top in my string (ie using sb.insert(0, stuff)). The "stuff" is known only I process the data from the ResultSet. Since I will be streaming the data directly (*without caching*) I will not be able to do that. Anyway I have to cope with that :) Will put the comment once I implement this idea. Thanks to you both!
0
 

Author Comment

by:javaq092999
ID: 7001184
Well this problem helped me in understanding the String and StringBuffer classes more closely and learned that how easily the StringBuffer could lead to "Memory Waste". I finally changed my most of the metods to stream the text to the JspWriter directly as I was using the same in many other places. Thanks all and specially "object".
0
 
LVL 92

Expert Comment

by:objects
ID: 7002568
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Introduction This article is the first of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article explains our test automation goals. Then rationale is given for the tools we use to a…
Viewers will learn about if statements in Java and their use The if statement: The condition required to create an if statement: Variations of if statements: An example using if statements:
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…
Suggested Courses

710 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question