Memory Usage with DataSets never reduces

lostinthegame
lostinthegame used Ask the Experts™
on
I have a vb.net windows form project that includes a dataset.  From my main form, the user can open a "load client" form that contains a datagridview.  When the "load client" form opens, I fill a dataset on the form from my sql ce database and then load the dataset into a datagridview.  The database I'm testing with has 10,000 records, and it causes the memory usage to skyrocket.  My issue is that when the user closes the form, the memory usage doesn't decrease.  When they open the "load client" form again, the memory usage almost doubles again.

The form closes using me.dispose, but it doesn't seem to have an effect on the memory usage.

How can I stop this from happening?  To me, it seems like the dataset fills, but the memory is never emptied, so each time I fill the dataset the memory just keeps increasing.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Commented:
Garbage collection won't run until certain thresholds are exceeded.  I know you said memory skyrockets, but it might not be enough.  To test this, add a couple of gc.collect calls to force garbage collection.  If this keeps the memory from increasing, you know nothing leaks and that is normal behavior.

Author

Commented:
That seems to work, the memory drops back down some now.  Do I need to continuously use gc.collect or just let it run as-is?

Without the garbage collector, the memory use goes from 70,000k to 200,000k on the first load, then to 400,000k.  If I repeatedly open and close the form, it climbs over a million.  I don't know if it makes a difference, but for these tests I have just been running the program in the debugger.

Author

Commented:
Also, I'm only using the task manager to measure the amount of memory that is being used.  From what I have found, that doesn't seem to be the best method, but I'm just trying to wrap my head around what's causing the memory usage to climb so high, and if it's an issue.
Top Expert 2015
Commented:
The GC (Garbage Collector) is the memory manager in .NET.

Just as you do with your own garbage, it does not go to the landfill everytime something needs to be destroyed (finalized is the right term to use in .NET). You wait until the garbage can is full before sending it to the landfill because it is the most efficient way to deal with garbage that accumulate in small chunks.

This is the same thing with memory management. Since finalizing an object requires work (thus time), and since you can have thousands of objects in memory (each column of a datatable is an object, each row is na object - and you have 10,000 - and the data in each cell is also an object. Calling the finalizer (the methode that cleans up the object) of all those objects when you get rid of the table would freeze the system for a while. The garbage collector thus wait until resources are low, and then can take decisions on the most efficient way of dealing with unused objects.

This thing actually optimize the use of memory. Unfortunately, Windows and .NET are 2 different worlds. Windows does not handle memory the same way and does not work with a Garbage Collector. The Task Manager in Windows only sees a huge chunk of memory reserved by .NET, but is not aware of what happens in that section of memory, so the reading you get there is not always representative the the actual resources in use when dealing with .NET applications.

As for calling GC.Collect, use it only as a diagnostic tool. Using it to clear memory can actually have a negative impact in the way the garbage collector works. It's akind to having cats playing in your garbage. The collection is not as efficient when cats go mess around your unused junk. Same fot the GC.

Commented:
Yes, there are some good articles out there explaining gc but this behavior is in the name of efficiency.  And they always recommend against routinely explicitly forcing gc like that.  It's been a while but I think there are several gc counters in Performance monitor (PerfMon) that you can watch to get a better handle on this behavior in your program.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial