tlsoftware
asked on
GlobalAlloc, LocalAlloc, new, malloc
Can anybody tell me what the difference between these are. For example what are the advantages are, if any, to using GlobalAlloc when I could just use new. And is there a difference between globalalloc and localalloc in windows 95.
// Choice 1
HGLOBAL h = GlobalAlloc(GMEM_MOVEABLE, 1000);
char* ptr = GlobalLock(h);
//do stuff with ptr
// Choice 2
HLOCAL l = LocalAlloc(LMEM_MOVABLE,10 00);
char *ptr = LocalLock(l);
// do stuff with ptr
// Choice 3
char *ptr = new char[1000];
// do stuff with ptr
// Choice 4
char *ptr = (char*)malloc(1000);
// do stuff with ptr
// Choice 1
HGLOBAL h = GlobalAlloc(GMEM_MOVEABLE,
char* ptr = GlobalLock(h);
//do stuff with ptr
// Choice 2
HLOCAL l = LocalAlloc(LMEM_MOVABLE,10
char *ptr = LocalLock(l);
// do stuff with ptr
// Choice 3
char *ptr = new char[1000];
// do stuff with ptr
// Choice 4
char *ptr = (char*)malloc(1000);
// do stuff with ptr
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I should mention that there is some slight overhead in using new over globalalloc, however the overhead is negligable compared to the length of the globalalloc call. So in a C or C++ program just use new.
Note in a debugger you can step into new and new [] and see how it works.
Hope this answers your question.
Note in a debugger you can step into new and new [] and see how it works.
Hope this answers your question.
Just to add some more confusion:
new and malloc() are usually implemented on a Win32 platform via yet another allocation API -- HeapAlloc(), which allows allocation from more than one heap.
There is also a VirtualAlloc() API that is used for efficient management of very large allocation requests.
An intelligent implementation of new or malloc() can allocate memory from different heaps (or using different methods) depending on the size of the allocation request and other parameters, thus reducing heap fragmentation and being more efficient than native APIs in the long run.
new and malloc() are usually implemented on a Win32 platform via yet another allocation API -- HeapAlloc(), which allows allocation from more than one heap.
There is also a VirtualAlloc() API that is used for efficient management of very large allocation requests.
An intelligent implementation of new or malloc() can allocate memory from different heaps (or using different methods) depending on the size of the allocation request and other parameters, thus reducing heap fragmentation and being more efficient than native APIs in the long run.
Heap framentation is not a big problem anymore. Virtual memory allows memory to be used efficiently even when the heap becomes fragmented. Actually, a highly fragmented heap still does have some drawbacks. For example, it may take longer to allocate memory from a very fragmented heap. This is because the allocation routine has to search through a lot of available fragments until it finds one of the right size. However, this process is still pretty fast, it is unussual for a program to have speed issues resulting from allocation.
The reason HeapAlloc is used is for cleanup purposes. The idea is when a program starts it allocates a block of memory from the operating system using GlobalAlloc. It then turns the block into a heap and allocates smaller chunks from that heap. If the program terminates without freeing all of its little allocated blocks (like from a crash or bad programming) it is not a problem. The clean-up routines or the operating system need to free only the one block of memory that was used to create the heap. Thus, there is no need to track every little block made the program, just the one heap.
I'm not with VirtualAlloc(). It looks like it and some of its companions can be used to manipulate blocks of virtual memory, that is force areas of memory in or out from the swap file. It is ussualy best to ignore the existance of virtual memory. Some programmers think they can speed up an algorithm that makes a lot of memory accesses if they prevent the OS from swaping out the memory. That tends not to be true, in fact it can slow things down. The operating system does a good of determining what should be in and what should be out. If you are touching a page a lot it won't swap it out. However, if you force it to keep memory swapped in that is not being used too much, it may be forced to swap out memory that is actually being used more. My feeling is that you can safely rely on the addvantages of virtual memory and ignore the dissadvatages. In addition, on today's 32 and 64 meg machines there is a lot less swapping than you might think. virtual memory is used primarily for rearranging physical memory and addresses, not so much for swapping.
The reason HeapAlloc is used is for cleanup purposes. The idea is when a program starts it allocates a block of memory from the operating system using GlobalAlloc. It then turns the block into a heap and allocates smaller chunks from that heap. If the program terminates without freeing all of its little allocated blocks (like from a crash or bad programming) it is not a problem. The clean-up routines or the operating system need to free only the one block of memory that was used to create the heap. Thus, there is no need to track every little block made the program, just the one heap.
I'm not with VirtualAlloc(). It looks like it and some of its companions can be used to manipulate blocks of virtual memory, that is force areas of memory in or out from the swap file. It is ussualy best to ignore the existance of virtual memory. Some programmers think they can speed up an algorithm that makes a lot of memory accesses if they prevent the OS from swaping out the memory. That tends not to be true, in fact it can slow things down. The operating system does a good of determining what should be in and what should be out. If you are touching a page a lot it won't swap it out. However, if you force it to keep memory swapped in that is not being used too much, it may be forced to swap out memory that is actually being used more. My feeling is that you can safely rely on the addvantages of virtual memory and ignore the dissadvatages. In addition, on today's 32 and 64 meg machines there is a lot less swapping than you might think. virtual memory is used primarily for rearranging physical memory and addresses, not so much for swapping.
Opps, it was supposed to read "I'm not familiar with VirtualAlloc." I'm sure there are plenty of other mistakes, but that leaped out at me.
VirtualAlloc() and friends can be useful for implementing sparse data structures.
For example, you can allocate memory for a huge spreadsheet but only commit the parts that are actually used. Basically, you can tell the OS to reserve a portion of the address space but not to map it to memory yet.
I agree that it is best to leave those low level details alone and let the OS manage them but they might come in useful in certain situations.
And regarding your comment on heap fragmentation, I think you are wrong on two accounts.
1) There are significant performance difference in allocators between compilers. Try compiling a program that allocates and frees randomly sized memory blocks in a random order under MSVC 2.x and 5.x and you'll see what I'm talking about. A magazine (DDJ I think) once benchmarked the allocators of several compilers and published the results. Also MS claimed a significant improvement (5x-10x) between compiler versions.
2) Virtual memory on intel arch. manages memory in 4K pages. Typical allocations happen to be smaller and have the annoying tendency of not falling on a page boundary.
For example, you can allocate memory for a huge spreadsheet but only commit the parts that are actually used. Basically, you can tell the OS to reserve a portion of the address space but not to map it to memory yet.
I agree that it is best to leave those low level details alone and let the OS manage them but they might come in useful in certain situations.
And regarding your comment on heap fragmentation, I think you are wrong on two accounts.
1) There are significant performance difference in allocators between compilers. Try compiling a program that allocates and frees randomly sized memory blocks in a random order under MSVC 2.x and 5.x and you'll see what I'm talking about. A magazine (DDJ I think) once benchmarked the allocators of several compilers and published the results. Also MS claimed a significant improvement (5x-10x) between compiler versions.
2) Virtual memory on intel arch. manages memory in 4K pages. Typical allocations happen to be smaller and have the annoying tendency of not falling on a page boundary.
Thanks for the info on virtual alloc. I coluld see where that could be very handy.
I was under the impression that most of the allocation work is handled by the OS. (In a release version.) The C++ compiler/implimentation just needs to figure out the space needed but the OS gets the space. I've certainly traced into new calls and have seen them make calls to the OS for getting memory.
Where fragmentation becomes an issue is when you allocate and dispose large blocks interspersed with allocating small blocks. This creates large gaps that are available seperated by small blocks in use. If you need to allocate a large block that is large than the current largest gap you are in trouble. However, with virtual memory it is not a problem because there small blocks can be be moved virtually. Yes, it could waste space because each of these small blocks might be a few bytes and use an entire 4 k page. That is not great, but at least the memory between the small blocks is available. (In a sense, the memory in between the small blocks is not even in existance, so memory elsewhere is available. Strange).
I was under the impression that most of the allocation work is handled by the OS. (In a release version.) The C++ compiler/implimentation just needs to figure out the space needed but the OS gets the space. I've certainly traced into new calls and have seen them make calls to the OS for getting memory.
Where fragmentation becomes an issue is when you allocate and dispose large blocks interspersed with allocating small blocks. This creates large gaps that are available seperated by small blocks in use. If you need to allocate a large block that is large than the current largest gap you are in trouble. However, with virtual memory it is not a problem because there small blocks can be be moved virtually. Yes, it could waste space because each of these small blocks might be a few bytes and use an entire 4 k page. That is not great, but at least the memory between the small blocks is available. (In a sense, the memory in between the small blocks is not even in existance, so memory elsewhere is available. Strange).
'Tis true, ultimately the OS manages memory allocation. However, the OS does not usually concern itself with fragmentation etc. issues so there is room for the compiler (actually, the library author) to optimize.
IIRC, MSVC (forget the version) manages two heaps - one for small allocations, another for large ones (the threshold settable at compile time).
I disagree however with your comment about virtual memory. Pages must be mapped in full or not at all. If the OS were to assign a page for each 4-byte allocation the memory usage would soar to the incridible 4K*(num of allocations). Granted this is virtual memory but imagine the disk thrashing resulting from the attempts of the poor OS to juggle those blocks.
Although today's OSes are quite a bit smarter, they can still use an occational help from the RTL.
[Off topic]
I know that Ex-Ex was not supposed to be a private discussion forum and I'm misusing it. OTOH, I cannot refrain from continuing an interesting discussion.
If you'd like to continue this (or any other) discussion after the question is graded, my email address is noted under my profile. Another possibility would be to open a 0-point question in a low traffic section and never grade it (or would that be considered abuse?).
IIRC, MSVC (forget the version) manages two heaps - one for small allocations, another for large ones (the threshold settable at compile time).
I disagree however with your comment about virtual memory. Pages must be mapped in full or not at all. If the OS were to assign a page for each 4-byte allocation the memory usage would soar to the incridible 4K*(num of allocations). Granted this is virtual memory but imagine the disk thrashing resulting from the attempts of the poor OS to juggle those blocks.
Although today's OSes are quite a bit smarter, they can still use an occational help from the RTL.
[Off topic]
I know that Ex-Ex was not supposed to be a private discussion forum and I'm misusing it. OTOH, I cannot refrain from continuing an interesting discussion.
If you'd like to continue this (or any other) discussion after the question is graded, my email address is noted under my profile. Another possibility would be to open a 0-point question in a low traffic section and never grade it (or would that be considered abuse?).
I didn't say that pages are not mapped in full. My only point is that the big problem with fragmentation is that it can lead to cases where there is enough TOTAL memory for a particular allocation, but there is no SINGLE block large enough. This has historically been a big problem. That is why some operating systems (Windows 3.1, Macintosh, some UNIXs) have moveble blocks. Movable blocks give the OS the oportunity to rearrange things so that big blocks are available. Unfortunately movable blocks, are an extra hastle for the programmer who must remember to lock and unlock them. Virtual memory provides the best of both worlds. The OS can rearrange things and the programmer does not have to go to any extra work. Yes you can get pages that are practically empty, but the next time you allocate something small it will go from one of those pages.
Why would you want to continue this conversation? Clearly I'm right and your wrong. The conversation is over <G>.
Actually, I don't think this conversation is a missuse. tlsoftware obviously wants to know the issues about dynamic memory management.
Why would you want to continue this conversation? Clearly I'm right and your wrong. The conversation is over <G>.
Actually, I don't think this conversation is a missuse. tlsoftware obviously wants to know the issues about dynamic memory management.
One issue you failed to address - disk thrashing.
Partially mapped pages certainly contribute to the phenomenon, so while the OS will protect you from the more destructive consequences of heap fragmentation, performance will suffer.
Well, I did get the impression that this was turning into a dialog. tlsoftware has been awfully quiet...
Partially mapped pages certainly contribute to the phenomenon, so while the OS will protect you from the more destructive consequences of heap fragmentation, performance will suffer.
Well, I did get the impression that this was turning into a dialog. tlsoftware has been awfully quiet...
Here we go again...
Yes, poorly filled pages will cause more swapping. However, I don't think that pages get that poorly filled. Subsequent small allocations will tend to fill them again.
Say you create a mix of large objects and small objects and delete the large objects. This leaves a bunch of small objects possibly as the only things on their pages. Definitly a waste, but is you program going to cause more swapping under this condition than under the previous condition? No, because it is now using less pages than before. (although they are poorly filled, there are still less of them.) Now if you start allocating again, the OS will tend to fill the pages again. So again things haven't gotten all that bad.
Now I don't want to sound overly optimistic. (And I don't want you to think for a second that I feel that anyone competant wrote the windows OS's) But by and large I don't think the typical program needs to concern itself with these issues. If you are doing some trully extensive allocation that needs to be accomplished quickly, maybe. But really I don't think that this is a bottleneck for most programs.
Yes, poorly filled pages will cause more swapping. However, I don't think that pages get that poorly filled. Subsequent small allocations will tend to fill them again.
Say you create a mix of large objects and small objects and delete the large objects. This leaves a bunch of small objects possibly as the only things on their pages. Definitly a waste, but is you program going to cause more swapping under this condition than under the previous condition? No, because it is now using less pages than before. (although they are poorly filled, there are still less of them.) Now if you start allocating again, the OS will tend to fill the pages again. So again things haven't gotten all that bad.
Now I don't want to sound overly optimistic. (And I don't want you to think for a second that I feel that anyone competant wrote the windows OS's) But by and large I don't think the typical program needs to concern itself with these issues. If you are doing some trully extensive allocation that needs to be accomplished quickly, maybe. But really I don't think that this is a bottleneck for most programs.
I agree with you that the programmer doesn't need to conccern himself with the matter. One of the reasons is that the library writer took some care of optimizations. You can check the sources that come with the compiler.
Good. Are we done? And where is tlsoftware? Decided not to wade though all this junk is suppose.
One thing I didn't see mentioned is that new calls an objects constructor where GlobalAlloc or VirtualAlloc won't. So it depends on what you are doing as to which you should call. You can also override the behavior of new so it could call GlobalAlloc or VirtualAlloc if needed.
New calls a constructor only if there is one (and new [] calls multiple times, if there is one). If the data being allocated is POD (plain old data. That's an official term, I didn't make it up.) then new just allocates memory and does not call any constructors. I was assuming the question meant the use of new to allocate POD.
char *Ptr = new char[1024];
In this case new is not conceptually different than the other calls, and in fact is pretty much mapped right to one of the other calls.
However as Greg pointed out, when allocating objects that have construtors, new is conceptially different and performs an important task. In these casses you must call new.
Similarly if an object has a destructor you must call delete, not one of the OS calls. In addition, new or delete should be paired. If you have something with no constructors but with a destructor, you must call delete for the destructor and you must call new because you will be calling delete.
char *Ptr = new char[1024];
In this case new is not conceptually different than the other calls, and in fact is pretty much mapped right to one of the other calls.
However as Greg pointed out, when allocating objects that have construtors, new is conceptially different and performs an important task. In these casses you must call new.
Similarly if an object has a destructor you must call delete, not one of the OS calls. In addition, new or delete should be paired. If you have something with no constructors but with a destructor, you must call delete for the destructor and you must call new because you will be calling delete.
I hope you are reading this tlsoftware. I would feel pretty silly if we're just talking amongst ourselves. Actually looking back at our "conversation", I feel pretty silly anyways.
ASKER
Yes I am reading it. I am totally absorbed. Thank you very very much for all the information.
Totally absorbed? You probably don't get out much. (Of course, I only left the house once this weekend and that was for groceries!)
Hi, All
I've get a program that receives data from host at 2k a time.
It eventually saves the data in paradox table.
The guy who wrote it (a contracter long gone) does many, many, many mallocs as he is converting the data for paradox etc...
So 600 transactions, converting to some 3000-10000 records, and allocation say 25 bytes for each field ...
Could the thrashing I am getting on the Win 95 machine running 24Mb, be attributed to what ever it was you have been talking about?
I could of course start a new Question and I will so have a look for it.
Zonnald
I've get a program that receives data from host at 2k a time.
It eventually saves the data in paradox table.
The guy who wrote it (a contracter long gone) does many, many, many mallocs as he is converting the data for paradox etc...
So 600 transactions, converting to some 3000-10000 records, and allocation say 25 bytes for each field ...
Could the thrashing I am getting on the Win 95 machine running 24Mb, be attributed to what ever it was you have been talking about?
I could of course start a new Question and I will so have a look for it.
Zonnald
I did 32 bytes per field (extra bytes for heap management data) times 5000 records and got 156K. That's not a lot of data to put in the heap. That total amount of data should not be enough to cause a lot of virtual memory swapping (disk trashing). The total number of objects however, seems like a lot (to me) and that could be a potential slow-down. But you specifically mentioned thrashing, so it probably isn't a problem.
Is this information being written to disk? That could be the cause. Especially if the information is beig written in small chunks, like one field at a time. Also if you disk has not been optimized recently and/or is getting full.
Is this information being written to disk? That could be the cause. Especially if the information is beig written in small chunks, like one field at a time. Also if you disk has not been optimized recently and/or is getting full.
I noticed this wasn't graded. Are you still there tlsoftware?
Now the C and C++ allocation procedures malloc and new are implimentation defined. They could be handled any way the people who wrote your version of C decides. However, for windows programming invironments, you can bet that new and malloc call GlobalAlloc to do the allocation. Typically, in fact, new calls malloc and malloc calls GlobalAlloc. Note that the pointer returned by new is not necessarily the pointer returned by Globalalloc though. New may allocate a slighly larger block than what should be needed so that it can store some additional information used to dispose of the block. It puts this information at the start of the block returned by GlobalAlloc and then returns a pointer to you that is past this information.