Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Memory allocation and disk thrashing.

Posted on 1998-01-08
4
Medium Priority
?
196 Views
Last Modified: 2010-08-05
Hi, All
I refer to a previous question regarding Global alloc et al.

I've get a program that receives data from host at 2k a time.

It eventually saves the data in paradox table.

The guy who wrote it (a contracter long gone) does many, many, many mallocs as he is converting the data for paradox etc...

So 600 transactions, converting to some 3000-10000 records, and allocation say 25 bytes for each field ...

Could the thrashing I am getting on the Win 95 machine running 24Mb, be attributed to what ever it was you have been talking about?


Zonnald
0
Comment
Question by:Zonnald
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 32

Accepted Solution

by:
jhance earned 200 total points
ID: 1177924
Yes, I would say that the above situation is very likely to cause thrashing.  It's very poor practice to malloc a lot of small blocks.  The problem is that malloc is not all that efficient and when allocating a small block it ends up wasting some space.  For a relatively small number of mallocs this is not a problem but when you do it thousands of times and put it in a loop with malloc's and free's you end up with a fragmented mess in memory.  In any kind of high volume transaction based program (like you have mentioned above) it's well worth the time and effort to build your own memory allocation utility which has a foreknowledge of the structure of the objects you are allocating and can do it much more eficiently.
0
 
LVL 22

Expert Comment

by:nietod
ID: 1177925
I dissagree.  (and would refer you back to the question that Zonnald is talking about if I could remember/find the QID.)

The total amount of heap space we are talking about here is on the order of a few hundred k.  The total number of allocations is a lot and may make for an innefficient heap, but the total size is not large enough to cause thrashing from virtual memory swapping.  Not these days, 5 years ago maybe. 10 years ago it was out of the question.

I suspect the problem is that the code is accessing the disk and causing the thrashing.  Perhaps it is writting out the information in small chuncks, which can be innefficient.


0
 
LVL 2

Expert Comment

by:anichini
ID: 1177926
If it is writing in small chunks, either buffer up large (i.e. at least 4 K) blocks of data or use memory-mapped files which does the same thing for you automatically (i'm simplifying).


0
 
LVL 1

Author Comment

by:Zonnald
ID: 1177927
It is writing the records to the disk via calls to Paradox.

I am not sure if it is doing several records at a time or if it is writing a record and closing the table.

Should this be the case perhaps we could open and close the database table only at the beginning and end of a particular set of records.

I will leave this open as there is a differing of opionions.  When we have a rewrite of the code which proves a praticular theory then I will sort out the points.

Zonnald
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Unlike C#, C++ doesn't have native support for sealing classes (so they cannot be sub-classed). At the cost of a virtual base class pointer it is possible to implement a pseudo sealing mechanism The trick is to virtually inherit from a base class…
Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
Suggested Courses

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question