Posted on 2003-12-08
Last Modified: 2007-12-19
I'm running a Java/J2EE/BC4J/JSP application on a sun box with Oracle 9i and 9iAS and getting a java.lang.OutOfMemoryError when accessing a large recordset a couple of times.  I read on Oracle and Sun websites that this error could be related to the Heap Size setting.  How can I find out what the current setting is and then change it?  What should they be set at?

Parameters: -Xms and -Xmx

Issac Rosa
Question by:irosa
LVL 24

Accepted Solution

shivsa earned 500 total points
ID: 9898504
To override Sun's JVM default memory setting use for example:

java -ms64m -mx96m ....

From the JDK documentation:
     -ms<number>       set the initial Java heap size
     -mx<number>       set the maximum Java heap size
LVL 24

Expert Comment

ID: 9898526
you can do is print the mem usage in some places using
RunTime rt = RunTime.getRunTime();

-ms64m -mx96m
meaning min 64 MB and max 96 MB
LVL 35

Expert Comment

ID: 9899122
> and getting a java.lang.OutOfMemoryError when accessing a large
>recordset a couple of times.  

  Can you post some code? You might have some flaws in your code that you will be able to avoid them? If not try shivsa's suggestion.

Author Comment

ID: 9899230
Here is the code for the JSP page.

<%@ page language="java" %>
<%@ page errorPage="/WEB-INF/jsp/Error.jsp" %>
<%@ taglib uri="/WEB-INF/lib/struts-html.tld" prefix="html" %>
<%@ taglib uri="/WEB-INF/lib/DataTags.tld" prefix="bc4j" %>

<!-- Security -->

<% //we explicitely set userCanWrite to true since user permissions will
   //be checked when we display the legal transition statuses
   // changed from SPROCE to SPROCB (Process Batches)
<% boolean userCanWrite = true; //false; %>
<%@ include file="CheckPermissions.txt" %>
<% userCanWrite = true; %>
<!-- /Security -->

<bc4j:ApplicationModule id="appBatchStatusTransition" configname="" releasemode="Stateful"/>
<bc4j:DataSource id="rowSetLegalStatusTransition" appid="appBatchStatusTransition" viewobject="LegalStatusTransitionsView"/>

<bc4j:ApplicationModule id="appModBatchProcessing" configname="" releasemode="Stateful"/>
<bc4j:DataSource id="rowSetBatches" appid="appModBatchProcessing" viewobject="BatchProcessingView"/>

  //Set the filter Criteria
  rowSetBatches.getRowSet().setWhereClauseParams((String[]) request.getSession().getAttribute("FilterCriteriaArray"));

      <title>Post Batches</title>
      <SCRIPT language="Javascript1.2" src="<html:rewrite forward='MenuScript'/>"></SCRIPT>
      <LINK href="<html:rewrite forward='IntranetStyle'/>" rel=stylesheet type=text/css>            

<form action="" method="post" name="frmPostBatch" id="frmPostBatch" target="_parent">

<table width="720" border="0" cellspacing="2" cellpadding="0" align="left" valign="top" bgcolor="#BEBEBE">
<!-- ** batch start ** -->
    int batchCount = 1;
    String curBatchId = "-1";
    String newBatchId = "0";
    boolean boolNewBatch = false;
    int rowCount = 0;
 <bc4j:RowsetIterate datasource="rowSetBatches">
       newBatchId = rowSetBatches.getRowSet().getCurrentRow().getAttribute("BatchInstanceId").toString();
       boolNewBatch = !curBatchId.equals(newBatchId);
       if (boolNewBatch) {
        <% if (!curBatchId.equals("-1")) { %>
        <% } %>
    <tr class="colhead2">
      <td width="400" class="tiny" align="left" onClick="changeMenu('tree.tables.batch<%= batchCount %>');" style="cursor:hand"><img src="images/mnuPlus.gif" alt="Click to View Cycles In Batch" name="tree.tables.batch<%= batchCount %>.image" id="mnuBatch<%= batchCount %>" border="0">&nbsp;<bc4j:ShowValue datasource="rowSetBatches" dataitem="BatchName"/></td>
      <td width="120" class="tiny" align="left"><strong><bc4j:ShowValue datasource="rowSetBatches" dataitem="StatusCodeDesc"/></strong></td>      
      <td width="200" class="tiny" align="left">
        <input type="hidden" name="BatchId" value="<bc4j:ShowValue datasource="rowSetBatches" dataitem="BatchInstanceId"/>">
        <input type="hidden" name="BatchName" value="<bc4j:ShowValue datasource="rowSetBatches" dataitem="BatchName"/>">        
              boolean showThisChoice = true;
              boolean userCanSee = true;
              int secureLegalTransitions = 0;
              //SysUserSecurityUtil securityUtil = new SysUserSecurityUtil();

           <select name="LegalTransitionStatusCode" size="1" class="tiny">                
                  String myMasterStatusCode = (String) rowSetBatches.getRowSet().getCurrentRow().getAttribute("CycleFileStatusCode");
                  //set filter criteria for detail
            <bc4j:RowsetIterate datasource="rowSetLegalStatusTransition">
                  String legalStatus = (String) rowSetLegalStatusTransition.getRowSet().getCurrentRow().getAttribute("ShortDescriptionLegalCode");
                  String legalStatusTransitionCode = (String) rowSetLegalStatusTransition.getRowSet().getCurrentRow().getAttribute("LegalTransitionStatusCode");

                  //if the user wants to reimport or reprocess, they'll have to use
                  //the cycle status processing screen. This allows them to change
                  //the status by file.
                  if (!(legalStatus == null || legalStatus.equals(""))) {                      
                     userCanSee = securityUtil.userShouldSeeStatus(request.getSession().getAttribute("userId").toString(), legalStatusTransitionCode);                  
                     if (userCanSee) {
                      <option value="<bc4j:ShowValue dataitem="LegalTransitionStatusCode"/>"><bc4j:ShowValue dataitem="ShortDescriptionLegalCode"/></option>
               <% }//end of check for userCanSee
                };//end check for null or ""
                 <!-- Handle Empty Combo Box Contents (due to Security) -->
                 <option value="-1" selected >
                      <%= (secureLegalTransitions != 0? "" : "Access Denied")  %>
    <!-- ** cycles in batch ** -->      
        <tr><td colspan=2>
          <div id="tree.tables.batch<%= batchCount %>b" style="display:none"></div>
          <div id="tree.tables.batch<%= batchCount %>">
            <table width="100%" border=0 cellspacing=0 cellpadding=0><script language="JavaScript">initMenu('tree.tables.batch<%= batchCount %>');</script>                  
       rowCount = 0;

               <tr class="tiny"><td width="10">&nbsp;</td><td align=left class="row<%=(rowCount % 2 + 1) %>" width="390"><img src="images/spacer.gif" width="16" height="1">&nbsp;&middot;&nbsp;<bc4j:ShowValue datasource="rowSetBatches" dataitem="ModuleCode"/>:&nbsp;<bc4j:ShowValue datasource="rowSetBatches" dataitem="CycleFileName"/>:&nbsp;<bc4j:ShowValue datasource="rowSetBatches" dataitem="CycleCode"/></td><td align="left" class="tiny"><bc4j:ShowValue datasource="rowSetBatches" dataitem="StatusCodeDesc"/></td></tr>      
       if (boolNewBatch) {
          curBatchId = newBatchId;
<!-- ** batch end ** -->      

<% request.getSession().removeAttribute("FilterCriteriaArray");%>

<bc4j:ReleasePageResources appid="appBatchStatusTransition" />
<bc4j:ReleasePageResources appid="appModBatchProcessing" />

Here is the code for the query in the bc4j component.
SELECT CellSites_EO.CELL_FACE_SECTOR,              CellSites_EO.CELL_SITE_ID,
       CellSites_EO.SIDBID,                    CellSites_EO.CELL_SITE_NAME,
       CellSites_EO.COMPANY_CODE,             CellSites_EO.HOME_COMPANY_PERCENTAGE,
       CellSites_EO.USID,                    CellSites_EO.CELL_FACE_LOCATION,
       CellSites_EO.IS_OVERRIDE,              CellSites_EO.INCURSION_COMPANY_1,
       CellSites_EO.INCURSION_COMPANY_2,        CellSites_EO.INCURSION_COMPANY_3,
       CellSites_EO.SERVING_PLACE,              CellSites_EO.DATA_SOURCE_FLAG,
       CellSites_EO.IS_LOCKED,                    CellSites_EO.LOCKED_BY_USER,
       CellSites_EO.LOCK_TIMESTAMP,              CellSites_EO.LAST_MODIFIED_USER_ID,
 first_company_sidbids.COMPANY_CODE default_home_company_code,      tb_city_cluster.city_csa_id,
       (select sidbid, MIN(company_code) company_code
        from company_sidbids
        group by sidbid) first_company_sidbids,              
       (select distinct company_sidbids.company_code, tb_csa_city.csa_id city_csa_id, tb_csa_market.parent_csa_id cluster_csa_id
       from company_sidbids, communication_service_areas tb_csa_city, communication_service_areas tb_csa_market
       where company_sidbids.company_code = tb_csa_city.company_code (+)
           and tb_csa_city.csa_type_name (+) = 'CITY'
       and tb_csa_city.parent_csa_id = tb_csa_market.csa_id (+)
       and tb_csa_market.csa_type_name (+) = 'MARKET') tb_city_cluster

WHERE CellSites_EO.sidbid = first_company_sidbids.sidbid (+)
  AND CellSites_EO.company_code = tb_city_cluster.company_code (+)
  AND (CellSites_EO.CELL_FACE_SECTOR like :0 or '-1' = :1)
  AND (CellSites_EO.CELL_SITE_ID like :2 or '-1' = :3)
  AND (CellSites_EO.SIDBID like :4 or '-1' = :5)
  AND (CellSites_EO.CELL_SITE_NAME like :6 or '-1' = :7)
  AND (CellSites_EO.COMPANY_CODE like :8 or '-1' = :9)
  AND (CellSites_EO.HOME_COMPANY_PERCENTAGE like :10 or '-1' = :11)
  AND (CellSites_EO.USID like :12 or '-1' = :13)
  AND (CellSites_EO.CELL_FACE_LOCATION like :14 or '-1' = :15)
  AND (CellSites_EO.IS_EXCLUDED like :16 or '-1' = :17)
  AND (CellSites_EO.IS_OVERRIDE like :18 or '-1' = :19)
  AND (CellSites_EO.INCURSION_COMPANY_1 like :20 or '-1' = :21)
  AND (CellSites_EO.INCURSION_COMPANY_2 like :22 or '-1' = :23)
  AND (CellSites_EO.INCURSION_COMPANY_3 like :24 or '-1' = :25)
  AND (CellSites_EO.INCURSION_COMPANY_1_PERCENT like :26 or '-1' = :27)
  AND (CellSites_EO.INCURSION_COMPANY_2_PERCENT like :28 or '-1' = :29)
  AND (CellSites_EO.INCURSION_COMPANY_3_PERCENT like :30 or '-1' = :31)
  AND (CellSites_EO.SWITCH_ID like :32 or '-1' = :33)
  AND (CellSites_EO.DATA_SOURCE_FLAG like :34 or '-1' = :35)
  AND (tb_city_cluster.cluster_csa_id like :36 or '-1' = :37)
  AND (tb_city_cluster.city_csa_id like :38 or '-1' = :39)
  AND (CellSites_EO.COMMENTS like :40 or '-1' = :41)

order by CellSites_EO.SIDBID, CellSites_EO.SWITCH_ID, CellSites_EO.CELL_SITE_ID, CellSites_EO.CELL_FACE_SECTOR
LVL 24

Expert Comment

ID: 9899272
could u just try to run this with more heap sizes as i suggested and see if it works.
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.


Expert Comment

ID: 9899473
Try using resultset instead of rowset. just make sure you close the connection after you done with it.
LVL 92

Expert Comment

ID: 9899500
what servlet container is 9ais using?

Expert Comment

ID: 9899791

remember that mx shouldn't be larger than your physical memory, and that setting them differently will allow the heap to shrink as well as grow, ie -ms64 -mx128 means the heap will grow to 128 under heavy usage but then the VM will garbage collect until the heap is back to 64, if it can. This can cause performance degradation if your app does a lot of work (lots of users, many threads etc) and you set the limits wide apart. Lots of objects will go through the create, use, discard cycle and the VM will spend more and more time garbage collecting and less time doing useful work.

In short, up the sizes, and set them both the same (then the VM won't want to start shrinking the heap). Make sure you close connections, dereference objects to allow them to be garbage collected and also try System.gc() to clean up before you do something you think is memory heavy. Bear in mind System.gc() is only a suggestion to the VM to garbage collect.

Here's something about perf. tuning bc4j, might give you some other pointers ...

Expert Comment

ID: 9902187
If the DATA is large enough, it will sooner or later throw an OutOfMemoryError, no matter how large the heap size.
As an alternative, try to load the data from the server in PAGES which are loaded according to the users current view.
Though this might slow the response time, it should solve the OutOfMemoryError.
LVL 35

Expert Comment

ID: 9902481
 I agree with javaTL, if the data is large enough you will eventually run out of memory. A few things:

a) Try to access the recordset only once and keep it in memory for further processing. Do not access it several times exept if the data changes all the time. If you do need to acees it several times make sure you set the previous variables to null.

b) as javaTL suggested it might be betetr to bring the results back in groups of (lets say) 100 and display them in separate pages to the user.

Expert Comment

ID: 9951237
I think your 'Out of Memory' might not be caused by a large resultset, unless you are working with an extremely large result set (more than 10K rows)  or an extremely small heap size (less than the default of 128M). Also, the part where you said 'large recordset a couple of times' leads me to guess that the large recordset works OK the first time, maybe even the second time. But if you keep requesting large recordsets, eventually you get the 'Out of Memory'

If this is the case, I would look for a resource that is not being returned to the system.

Here is a 'back of the envelope' calculation.

You are selecting about 25 columns, many of which are simple numbers. So lets say each column takes an average of about 100 bytes.

That means each row takes 2.5k. Let's say that the wrapper objects need another 2.5k, giving 5k per row.

So if this is true, 200 rows would require 1 M of heap spece. (It might be as many as 2000, because I am trying to make a pessimistic estimate. Also, for the columns that have String values, identical columns will eventually share memory)

Now the JSP and Servlet container will need some heap space to operate. Let's say it needs 10 - 20 MB, + maybe another 5 - 10 MB per concurrent request. I believe the default heap size is usually 128 MB, so that leaves about maybe 100 MB for the query data.

This means that your page should be able to accomadate about as many as 1/4 million rows, but certainly at least 10,000 rows (if only a single user uses your system. If more than one request is being processed at a time, you can split the 1/4 million among all concurrent requests).

And... if your query can return 10,000 rows, and you want to display all that on a single web page... It's like JavaTL suggested. You might want to add paging to your design.

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Eclipse Neon and jdk 1.8.0 11 138
Starting to learn JAVA, 7 59
javap bin 2 29
How  Can  I  resolve   this  Error "java.sql.SQLException: Invalid column index" 4 25
For customizing the look of your lightweight component and making it look lucid like it was made of glass. Or: how to make your component more Apple-ish ;) This tip assumes your component to be of rectangular shape and completely opaque. (COD…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now