Solved

Memory usuage is increasing  drastically on using StringBUffer???

Posted on 2002-03-07
21
343 Views
Last Modified: 2013-12-29
I am making an RMI call to get the contents of the file in the form of StringBuffer to display that in the text Area.
When i receive the stringBuffer in my UI, its size is nearly 6 times that of the original file. i.e. if the file size is 2mb
StringBuffer size increase to around 12-13 mb. I am creating the buffer like this

public StringBuffer getBuffer()
{
STringBuffer strFileContents = new StringBuffer();
     File fSgFile = new File(name);
    FileReader fReadder = new FileReader(fSgFile);
    BufferedReader bReader          = new BufferedReader(fReadder);
   String strLine;
    while (  (strLine=bReader.readLine()) != null)
    {
     //     APPEND THE LINE TO THE BUFFER
     strFileContents.append(strLine);
     
     strFileContents.append( "\n" )
 }
return strFileContents;
}


Further add to the problem is JTextArea.When I add this StringBuffer to the text Area. Memory usuage again increases
drastically.

How can I reduce these things.
0
Comment
Question by:ugarg
  • 7
  • 4
  • 3
  • +4
21 Comments
 
LVL 4

Expert Comment

by:m_onkey_boy
ID: 6849568
INstead of double-buffering in the reader and stringbuffer, can you return the reader instead?
0
 
LVL 9

Expert Comment

by:Venci75
ID: 6850073
try this:
public StringBuffer getBuffer()
{
  File fSgFile = new File(name);
  StringBuffer strFileContents = new StringBuffer(fSgFile.length());
...
0
 

Author Comment

by:ugarg
ID: 6851001
I tried using the above, but it didn't help, but when i created the stringbuffer with
some lesser initialsize( than the file size), than it slightly reduuces the memory usuage.
but its very marginal improvement
0
 

Author Comment

by:ugarg
ID: 6851005
I cann't use Reader as it is not serialized.
0
 
LVL 7

Expert Comment

by:Igor Bazarny
ID: 6854557
Hi,

Try to use String instead of StringBuffer. My guess is that StringBuffer is not designed for compactness. BTW, StringBuffer size should be at least twice more than file size, because in typical case one byte corresponds to one 2-byte character. It could be that during serialization StringBuffer reserved size is doubled (it's expected that StringBuffer will increase in size, so code prepares a room for expansion).

Try to clear all references to received String[Buffer] and let garbage collector to clean used memory--JTextArea makes copy of your text.  

Regards,
Igor Bazarny,
Brainbench MVP for Java 1
www.brainbench.com
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 6854941
bazarny's idea is a good one. When you append to a StringBuffer, it doesn't merely grow by the size of the data appended to it; it grows exponentially! This is why you are finding that a huge amount of data is coming over the wire. The tragedy is that most of it is null characters at the end of the buffer. So you've two solutions:
1. Return a String as bazarny suggests
2. If you want to manipulate the data using the StringBuffer at the other end of the wire, call strFileContents.setLength(strFileContents.length())

Of course, if you don't *need* to use the StringBuffer, then returning the String will represent less overhead.
0
 
LVL 4

Expert Comment

by:m_onkey_boy
ID: 6855159
Appending to a String over and over again is HUGELY innefficient - even worse than the StringBuffer.  I'd stick with the SringBuffer(file.size() * 2);  This shouldn't allocate anything it doesn't need.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 6855309
Just to clarify my earlier comment, by using a String instead of a StringBuffer, I mean to *return* a string, not to start out with a String and append to it.

In point of fact appending to a String by concatenation is compiled to appending to a StringBuffer and the resulting instructions are not hugely different (see below)

You may as well leave your code as it is but ensure that you only return the information you need by calling strFileContents.setLength(strFileContents.length()) before you return the StringBuffer contents. This will effectively trim the StringBuffer's buffer of the vast number of null characters, returning only the ones you need.

-------------------------------------------------------
import java.io.*;

public class StringAppend {

  public String getBufferAsString(String name) throws Exception {
    String wholeFile = null;
    File fSgFile = new File(name);
    FileReader fReadder = new FileReader(fSgFile);
    BufferedReader bReader = new BufferedReader(fReadder);
    String strLine;
    while ((strLine = bReader.readLine()) != null) {
      //APPEND THE LINE TO THE BUFFER
      wholeFile += strLine + "\n";
    }
    return wholeFile;
  }

  public StringBuffer getBuffer(String name) throws Exception {
    StringBuffer strFileContents = new StringBuffer();
    File fSgFile = new File(name);
    FileReader fReadder = new FileReader(fSgFile);
    BufferedReader bReader = new BufferedReader(fReadder);
    String strLine;
    while ((strLine = bReader.readLine()) != null) {
      //APPEND THE LINE TO THE BUFFER
      strFileContents.append(strLine);
      strFileContents.append("\n");
    }
    return strFileContents;
  }

}

DECOMPILES TO:

Compiled from StringAppend.java
public class StringAppend extends java.lang.Object {
    public StringAppend();
    public java.lang.String getBufferAsString(java.lang.String) throws java.lang.Exception;
    public java.lang.StringBuffer getBuffer(java.lang.String) throws java.lang.Exception;
}

Method StringAppend()
   0 aload_0
   1 invokespecial #1 <Method java.lang.Object()>
   4 return

Method java.lang.String getBufferAsString(java.lang.String)
   0 aconst_null
   1 astore_2
   2 new #2 <Class java.io.File>
   5 dup
   6 aload_1
   7 invokespecial #3 <Method java.io.File(java.lang.String)>
  10 astore_3
  11 new #4 <Class java.io.FileReader>
  14 dup
  15 aload_3
  16 invokespecial #5 <Method java.io.FileReader(java.io.File)>
  19 astore 4
  21 new #6 <Class java.io.BufferedReader>
  24 dup
  25 aload 4
  27 invokespecial #7 <Method java.io.BufferedReader(java.io.Reader)>
  30 astore 5
  32 goto 60
  35 new #8 <Class java.lang.StringBuffer>
  38 dup
  39 invokespecial #9 <Method java.lang.StringBuffer()>
  42 aload_2
  43 invokevirtual #10 <Method java.lang.StringBuffer append(java.lang.String)>
  46 aload 6
  48 invokevirtual #10 <Method java.lang.StringBuffer append(java.lang.String)>
  51 ldc #11 <String "">
  53 invokevirtual #10 <Method java.lang.StringBuffer append(java.lang.String)>
  56 invokevirtual #12 <Method java.lang.String toString()>
  59 astore_2
  60 aload 5
  62 invokevirtual #13 <Method java.lang.String readLine()>
  65 dup
  66 astore 6
  68 ifnonnull 35
  71 aload_2
  72 areturn

Method java.lang.StringBuffer getBuffer(java.lang.String)
   0 new #8 <Class java.lang.StringBuffer>
   3 dup
   4 invokespecial #9 <Method java.lang.StringBuffer()>
   7 astore_2
   8 new #2 <Class java.io.File>
  11 dup
  12 aload_1
  13 invokespecial #3 <Method java.io.File(java.lang.String)>
  16 astore_3
  17 new #4 <Class java.io.FileReader>
  20 dup
  21 aload_3
  22 invokespecial #5 <Method java.io.FileReader(java.io.File)>
  25 astore 4
  27 new #6 <Class java.io.BufferedReader>
  30 dup
  31 aload 4
  33 invokespecial #7 <Method java.io.BufferedReader(java.io.Reader)>
  36 astore 5
  38 goto 55
  41 aload_2
  42 aload 6
  44 invokevirtual #10 <Method java.lang.StringBuffer append(java.lang.String)>
  47 pop
  48 aload_2
  49 ldc #11 <String "">
  51 invokevirtual #10 <Method java.lang.StringBuffer append(java.lang.String)>
  54 pop
  55 aload 5
  57 invokevirtual #13 <Method java.lang.String readLine()>
  60 dup
  61 astore 6
  63 ifnonnull 41
  66 aload_2
  67 areturn
0
 
LVL 7

Expert Comment

by:Igor Bazarny
ID: 6855736
Hi,

> SringBuffer(file.size() * 2);  
Well, StringBuffer size counted in chars, so there is no need to double it.

CEHJ is right, I didn't mean that String should be used during read operation, but I feel that serializing String would be more efficient than serializing StringBuffer of the same size, especially when StringBuffer allocated a bit of extra space. BTW, 'exponential' growth is not really bad when you think about price of moving content to the new place during reallocation. Doubling size each time current capacity exceeded results in log(size) reallocations (with about size copy operations) and at most 50% of wasted memory.

Regards,
Igor Bazarny
0
 
LVL 9

Expert Comment

by:Venci75
ID: 6856448
Have you tried to call System.gc()?
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 86

Expert Comment

by:CEHJ
ID: 6856504
That wouldn't help, Venci75. The problem is the vastly swollen buffer inside the StringBuffer, which wouldn't get garbage-collected.

Experiment with the initial size of the StringBuffer as bazarny suggested and then make the call I suggested before returning.
0
 
LVL 9

Expert Comment

by:Venci75
ID: 6856531
Look - I was the first one that suggested to use the initial size of the StringBuffer.
But I think that the StringBuffer is not the only problem - there is a lot of garbage generated by the BufferedReader.
So - if ugarg really have to use BufferedReader, instead of reading the file with a buffer, which size won't be increased latter - may be the memory allocated by the BufferedReader should be released.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 6856564
That wouldn't help, Venci75. The problem is the vastly swollen buffer inside the StringBuffer, which wouldn't get garbage-collected.

Experiment with the initial size of the StringBuffer as bazarny suggested and then make the call I suggested before returning.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 6856575
Yes, Venci75 you were the first one, sorry! But let's have the questioner do some practical tests before we go any further with theory! (s/he's not even replied yet!)
0
 
LVL 7

Expert Comment

by:Igor Bazarny
ID: 6856626
Hi,

If I understand correctly, the problem is on receiving side of RMI call which is not in JVM where all BufferedReader stuff happens. So I was trying to suggest ways to improve serialization performance.

But let's wait for questioner. ugarg, do you have more questions? Did you get your problem solved?

Igor
0
 

Author Comment

by:ugarg
ID: 6857443
Now instead of passing as StringBuffer I am passing the file contents in the form
of byte array and on the recieving end, i constructed String object from it like

String s = new String( byteArray );

However this  way also memory is sharply rising( ard 6 times as that of actual file size)
and memory usuage increases during string creation only. I couldn't notice much difference by not using String Buffer.

I am creating Byte Array like

         byte data[]
         File file = new File( name );
        data = new byte[ (int)(file.length()) ];
                  (new FileInputStream( file )).read( data )

I will try with whatever CEHJ has suggested         i.e. strFileContents.setLength(strFileContents.length())
and will let u guys know.

Thanks

0
 
LVL 1

Expert Comment

by:Moondancer
ID: 6961492
ADMINISTRATION WILL BE CONTACTING YOU SHORTLY.  Moderators Computer101 or Netminder will return to finalize these if they are still open in 14 days.  Experts, please post closing recommendations before that time.

Below are your open questions as of today.  Questions which have been inactive for 21 days or longer are considered to be abandoned and for those, your options are:
1. Accept a Comment As Answer (use the button next to the Expert's name).
2. Close the question if the information was not useful to you, but may help others. You must tell the participants why you wish to do this, and allow for Expert response.  This choice will include a refund to you, and will move this question to our PAQ (Previously Asked Question) database.  If you found information outside this question thread, please add it.
3. Ask Community Support to help split points between participating experts, or just comment here with details and we'll respond with the process.
4. Delete the question (if it has no potential value for others).
   --> Post comments for expert of your intention to delete and why
   --> YOU CANNOT DELETE A QUESTION with comments; special handling by a Moderator is required.

For special handling needs, please post a zero point question in the link below and include the URL (question QID/link) that it regards with details.
http://www.experts-exchange.com/jsp/qList.jsp?ta=commspt
 
Please click this link for Help Desk, Guidelines/Member Agreement and the Question/Answer process.  http://www.experts-exchange.com/jsp/cmtyHelpDesk.jsp

Click you Member Profile to view your question history and please keep them updated. If you are a KnowledgePro user, use the Power Search option to find them.  

Questions which are LOCKED with a Proposed Answer but do not help you, should be rejected with comments added.  When you grade the question less than an A, please comment as to why.  This helps all involved, as well as others who may access this item in the future.  PLEASE DO NOT AWARD POINTS TO ME.

To view your open questions, please click the following link(s) and keep them all current with updates.
http://www.experts-exchange.com/questions/Q.20260149.html
http://www.experts-exchange.com/questions/Q.20260294.html
http://www.experts-exchange.com/questions/Q.20274743.html
http://www.experts-exchange.com/questions/Q.20274749.html
http://www.experts-exchange.com/questions/Q.20281226.html
http://www.experts-exchange.com/questions/Q.20282947.html



*****  E X P E R T S    P L E A S E  ******  Leave your closing recommendations if this item.  If you are interested in the cleanup effort, please click this link http://www.experts-exchange.com/jsp/qManageQuestion.jsp?ta=commspt&qid=20274643
POINTS FOR EXPERTS awaiting comments are listed here -> http://www.experts-exchange.com/commspt/Q.20277028.html
 
Moderators will finalize this question if in @14 days Asker has not responded.  This will be moved to the PAQ (Previously Asked Questions) at zero points, deleted or awarded.
 
Thanks everyone.
Moondancer
Moderator @ Experts Exchange
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 6962173
To be more precise, in one of my earlier posts, I used the word 'exponentially' to describe how the StringBuffer expands when allocation is insufficient. This is not true in the mathematical sense, as doubling anything is not applying an exponential function!
0
 
LVL 9

Expert Comment

by:Venci75
ID: 7902199
No comment has been added lately, so it's time to clean up this TA.
I will leave a recommendation in the Cleanup topic area that this question is:
PAQ'd and pts refunded
Please leave any comments here within the next seven days.
 
PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!
 
Venci75
EE Cleanup Volunteer
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 7902688
OK...
0
 
LVL 5

Accepted Solution

by:
Netminder earned 0 total points
ID: 7962496
Per recommendation, points refunded and question closed.

Netminder
EE Admin
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
This video teaches viewers about errors in exception handling.

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now