Solved

Memory allocation and disk thrashing.

Posted on 1998-01-08
4
188 Views
Last Modified: 2010-08-05
Hi, All
I refer to a previous question regarding Global alloc et al.

I've get a program that receives data from host at 2k a time.

It eventually saves the data in paradox table.

The guy who wrote it (a contracter long gone) does many, many, many mallocs as he is converting the data for paradox etc...

So 600 transactions, converting to some 3000-10000 records, and allocation say 25 bytes for each field ...

Could the thrashing I am getting on the Win 95 machine running 24Mb, be attributed to what ever it was you have been talking about?


Zonnald
0
Comment
Question by:Zonnald
4 Comments
 
LVL 32

Accepted Solution

by:
jhance earned 100 total points
ID: 1177924
Yes, I would say that the above situation is very likely to cause thrashing.  It's very poor practice to malloc a lot of small blocks.  The problem is that malloc is not all that efficient and when allocating a small block it ends up wasting some space.  For a relatively small number of mallocs this is not a problem but when you do it thousands of times and put it in a loop with malloc's and free's you end up with a fragmented mess in memory.  In any kind of high volume transaction based program (like you have mentioned above) it's well worth the time and effort to build your own memory allocation utility which has a foreknowledge of the structure of the objects you are allocating and can do it much more eficiently.
0
 
LVL 22

Expert Comment

by:nietod
ID: 1177925
I dissagree.  (and would refer you back to the question that Zonnald is talking about if I could remember/find the QID.)

The total amount of heap space we are talking about here is on the order of a few hundred k.  The total number of allocations is a lot and may make for an innefficient heap, but the total size is not large enough to cause thrashing from virtual memory swapping.  Not these days, 5 years ago maybe. 10 years ago it was out of the question.

I suspect the problem is that the code is accessing the disk and causing the thrashing.  Perhaps it is writting out the information in small chuncks, which can be innefficient.


0
 
LVL 2

Expert Comment

by:anichini
ID: 1177926
If it is writing in small chunks, either buffer up large (i.e. at least 4 K) blocks of data or use memory-mapped files which does the same thing for you automatically (i'm simplifying).


0
 
LVL 1

Author Comment

by:Zonnald
ID: 1177927
It is writing the records to the disk via calls to Paradox.

I am not sure if it is doing several records at a time or if it is writing a record and closing the table.

Should this be the case perhaps we could open and close the database table only at the beginning and end of a particular set of records.

I will leave this open as there is a differing of opionions.  When we have a rewrite of the code which proves a praticular theory then I will sort out the points.

Zonnald
0

Featured Post

ScreenConnect 6.0 Free Trial

Discover new time-saving features in one game-changing release, ScreenConnect 6.0, based on partner feedback. New features include a redesigned UI, app configurations and chat acknowledgement to improve customer engagement!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
thread-safe code in c++ 2 102
Least Squares Curve Fitting 4 78
Create a path if not exists 7 77
How to read a c++ line with strings and variable numbers of elements 6 27
Templates For Beginners Or How To Encourage The Compiler To Work For You Introduction This tutorial is targeted at the reader who is, perhaps, familiar with the basics of C++ but would prefer a little slower introduction to the more ad…
Introduction This article is a continuation of the C/C++ Visual Studio Express debugger series. Part 1 provided a quick start guide in using the debugger. Part 2 focused on additional topics in breakpoints. As your assignments become a little more …
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.
The viewer will be introduced to the member functions push_back and pop_back of the vector class. The video will teach the difference between the two as well as how to use each one along with its functionality.

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question