Link to home
Start Free TrialLog in
Avatar of annapolistech
annapolistechFlag for United States of America

asked on

java.lang.OutOfMemoryError: Java heap space for data (retrieved from database) added to arraylist

I am populating a bean using the data retieved from database by executing a qyery like "SELECT * FROM xyz", where the total rows returned by the query is about 16000. Finally I add the bean to an ArrayList and storing the list in session, a custom tag in another JSP will  scroll through the list to display paginated results.
The problem is, as soon as I add the bean to the arraylist, I am getting a java.lang.OutOfMemoryError: Java heap space exception which in turn is stopping the flow of execution.

Any help in this regard will be appreciated,
Thanks

Avatar of ksivananth
ksivananth
Flag of United States of America image

make sure you are not making any copy of the bean and if it doesn't help increase the heap!
SOLUTION
Avatar of arevuri
arevuri
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
It is recommend to use same value for both minium and maximum to avoid performance issues. 512m above indicates 512 mega bytes of heap memory.
It seems you have pagination already. hence write logic to get only amount of data that gets displayed in UI. For example if you are displaying 10 records in each page, you do not need to get all 16000 record at a time and filter in your java, you can write query to get only required 10 records for that particual page
 
Avatar of annapolistech

ASKER

Thaks for all your responses.

We are using netbeans for development, and I am not preetry sure abt the place where I need to increase -Xmx and -Xms parameter values in netbeans. Even if I find that out, would that be a feasable solution.

We are using Display tag library for the pagination purpose where in we pass the entire resultset in the form of an array list for iteration.

As far as partial data retrieval is concern (10/20 records at a time), then I guess everytime I need different volume of data, I need to access the DAO layer to execute the query accordingly, which I doubt as performance issue.

Please point me if  I am wrong, and help me with a feasable solution.
Below site gives you good information on JVM parameters for Netbeans
                http://performance.netbeans.org/howto/jvmswitches
 
Getting 16000 records at a time and keep them in memory is performance issue than getting x (10 or 20) number of records from database when ever user requested. Getting minimal records would not be a performance issue w.r.t of memory , but there could be multiple hits to the server.
Additionally you can proivde some filter criteria where user can filter the data as required
If end user needs all the records, then good idea is to generate excel reports and FTP (or SFTP) them to some FTP location.
 
All you need to do is validate the exact requirement. if end user needs the entire 16000 records, then creat a file and download the file into user area. If user needs to just see the records then you can provide filter criteria and pagination.
Hi,

As  we mentioned that we are using Display tag library for the pagination purpose. Is there a way we can use the tag to get x (10 or 20) number of records from database when ever user requested. If not using the tag, then some other way.

Thanks
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hi,

We are using MYSQL.
The problem with the Display tag is, it doesn't interact with the DB, it just iterate thru the list of data (e.g. Arraylist) returned by the query. So it becomes hard to tell the query to fetch 'n'  number of records.

Thanks


I believe you must be sending some input values to get data, send your page number along with that input data every time page is clicked, if 2nd Page is clicked show from 11 to 20th record if 3rd page is clicked show from 21 to 30 record
 
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial