[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 649
  • Last Modified:

java.lang.OutOfMemoryError: Java heap space for data (retrieved from database) added to arraylist

I am populating a bean using the data retieved from database by executing a qyery like "SELECT * FROM xyz", where the total rows returned by the query is about 16000. Finally I add the bean to an ArrayList and storing the list in session, a custom tag in another JSP will  scroll through the list to display paginated results.
The problem is, as soon as I add the bean to the arraylist, I am getting a java.lang.OutOfMemoryError: Java heap space exception which in turn is stopping the flow of execution.

Any help in this regard will be appreciated,
Thanks

0
annapolistech
Asked:
annapolistech
  • 7
  • 4
4 Solutions
 
ksivananthCommented:
make sure you are not making any copy of the bean and if it doesn't help increase the heap!
0
 
arevuriCommented:
increase -Xmx and -Xms parameter values in your server start up/configuration script against java command. If these parameters are not available then add those
 
Example: java -Xms 512m -Xmx -512
-Xms is minimum an d-Xmx is maximum heap size to be allocated.
 
Note: - It is not recommend to read huge data into memory. If you require this data for UI report then display the report by getting 10 or 20 or little more records each time and provide pagination to display records in next pages.
0
 
arevuriCommented:
It is recommend to use same value for both minium and maximum to avoid performance issues. 512m above indicates 512 mega bytes of heap memory.
It seems you have pagination already. hence write logic to get only amount of data that gets displayed in UI. For example if you are displaying 10 records in each page, you do not need to get all 16000 record at a time and filter in your java, you can write query to get only required 10 records for that particual page
 
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
annapolistechAuthor Commented:
Thaks for all your responses.

We are using netbeans for development, and I am not preetry sure abt the place where I need to increase -Xmx and -Xms parameter values in netbeans. Even if I find that out, would that be a feasable solution.

We are using Display tag library for the pagination purpose where in we pass the entire resultset in the form of an array list for iteration.

As far as partial data retrieval is concern (10/20 records at a time), then I guess everytime I need different volume of data, I need to access the DAO layer to execute the query accordingly, which I doubt as performance issue.

Please point me if  I am wrong, and help me with a feasable solution.
0
 
arevuriCommented:
Below site gives you good information on JVM parameters for Netbeans
                http://performance.netbeans.org/howto/jvmswitches
 
0
 
arevuriCommented:
Getting 16000 records at a time and keep them in memory is performance issue than getting x (10 or 20) number of records from database when ever user requested. Getting minimal records would not be a performance issue w.r.t of memory , but there could be multiple hits to the server.
Additionally you can proivde some filter criteria where user can filter the data as required
If end user needs all the records, then good idea is to generate excel reports and FTP (or SFTP) them to some FTP location.
 
All you need to do is validate the exact requirement. if end user needs the entire 16000 records, then creat a file and download the file into user area. If user needs to just see the records then you can provide filter criteria and pagination.
0
 
annapolistechAuthor Commented:
Hi,

As  we mentioned that we are using Display tag library for the pagination purpose. Is there a way we can use the tag to get x (10 or 20) number of records from database when ever user requested. If not using the tag, then some other way.

Thanks
0
 
arevuriCommented:
You need to write your database query in such a way to get 10 or 20 records each time, use rownum (to get 1 to 10 , 11 to 20 like that) and sorting criteria in the query in your database layer and push the data to your taglib
 
Which databse you are using?
0
 
annapolistechAuthor Commented:
Hi,

We are using MYSQL.
The problem with the Display tag is, it doesn't interact with the DB, it just iterate thru the list of data (e.g. Arraylist) returned by the query. So it becomes hard to tell the query to fetch 'n'  number of records.

Thanks


0
 
arevuriCommented:
I believe you must be sending some input values to get data, send your page number along with that input data every time page is clicked, if 2nd Page is clicked show from 11 to 20th record if 3rd page is clicked show from 21 to 30 record
 
0
 
arevuriCommented:
Otherwise write your own customized taglib for this kind of functioality using JSP Tag Libraries
0
 
annapolistechAuthor Commented:
Although I have tried all the solutions sent by the experts, I am unable to fix the problem.
I'll figure out the solution so I am  closing the question.
Thanks
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

  • 7
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now