GlobalAlloc - where does the memory come from?

I'm looking for an explanation of where the memory returned from GlobalAlloc comes from. Based on the fact that this memory can be accessed across module and process boundaries, it seems that this must be a system-wide heap that is mapped into the same virtual address for all processes. Is this essentially how it works?

If this is the case, then the pages that malloc allocates from the OS must come from a different heap since those addresses are not valid across modules. However, when reading about heap management in win32, it looks like there are *not* two different heaps (though back in Win 3.1 there used to be the docs say).
LVL 4
jimstarAsked:
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

x
 
jkrConnect With a Mentor Commented:
>>Does Windows pre-allocate a chunk of memory for use as the heap at startup

Yes, for each process a heap is created. From the same book:

"Every process starts out with a default process heap, usually 1 MB in size (unless specified otherwise in the image file by using the /HEAP linker flag). This size is just the initial reserve, howeverit will expand automatically as needed. (You can also specify the initial committed size in the image file.) Win32 applications as well as several Win32 functions that might need to allocate temporary memory blocks use this process default heap. Processes can also create additional private heaps with the HeapCreate function. When a process no longer needs a private heap, it can recover the virtual address space by calling HeapDestroy. Only a private heap created with HeapCreatenot the default heapcan be destroyed during the life of a process.

To allocate memory from the default heap, a thread must obtain a handle to it by calling GetProcessHeap. (This function returns the address of the data structure that describes the heap, but callers should never rely on that.) A thread can then use the heap handle in calls to HeapAlloc and HeapFree to allocate and free memory blocks from that heap. The heap manager also provides an option for each heap to serialize allocations and deallocations so that multiple threads can call heap functions simultaneously without corrupting heap data structures. The default process heap is set to have this serialization by default (though you can override this on a call-by-call basis). For additional private heaps, a flag passed to HeapCreate is used to specify whether serialization should be performed."

http://book.itzero.com/read/microsoft/0507/microsoft.press.microsoft.windows.internals.fourth.edition.dec.2004.internal.fixed.ebook-ddu_html/0735619174/ch07lev1sec2.html ("Services the Memory Manager Provides")

>>Unless I'm mistaken, Windows would need to keep process-specific heap
>>memory

That is what Windows does...

>>This would seem to mean it has several pages dedicated to
>>globally-accessible heap data

No, that is not the case (as opposed to kernel space, but there, that's threads). From the above: "Shared memory can be defined as memory that is visible to more than one process or that is present in more than one process virtual address space. For example, if two processes use the same DLL, it would make sense to load the referenced code pages for that DLL into physical memory only once and share those pages between all processes that map the DLL"

These pages are simply mapped into more than one process' address space, but there is no longer any "global pool" on NT based systems (as opposed to Win9x-based ones)
0
 
Anthony2000Connect With a Mentor Commented:
Not my understanding. The pages from malloc are taken from the same heap as GlobalAlloc. I am writing this based on what the docs have written.
0
 
Anthony2000Commented:
Here is a link which discusses the topic in greater detail:

http://msdn2.microsoft.com/en-us/library/ms810603.aspx
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
jkrConnect With a Mentor Commented:
>>Based on the fact that this memory can be accessed across module and
>>process boundaries, it seems that this must be a system-wide heap that is
>>mapped into the same virtual address for all processes.

The system's memory manager takes care of transforming the available physical memory (plus swap) on a machine into virtual 4GB-sized address spaces for the running processes. This is done using "page lists" that show which process is using a certain page and whether it resides on disk or in memory. Mark Russinowich's "Inside Windows 2000" sums that up as

"As is true with most modern operating systems, Windows 2000 provides a mechanism to share memory among processes and the operating system. Shared memory can be defined as memory that is visible to more than one process or that is present in more than one process virtual address space. For example, if two processes use the same DLL, it would make sense to load the referenced code pages for that DLL into physical memory only once and share those pages between all processes that map the DLL" [...]
"User applications reference 32-bit virtual addresses. Using data structures the memory manager creates and maintains, the CPU translates virtual addresses into physical addresses.[...]"


The same applies for allocating pages using 'GlobalAlloc()'. For more, see http://book.itzero.com/read/microsoft/0507/microsoft.press.microsoft.windows.internals.fourth.edition.dec.2004.internal.fixed.ebook-ddu_html/0735619174/ch07lev1sec5.html
0
 
jimstarAuthor Commented:
Thanks for your comments. A few clarifications:

1. Does Windows pre-allocate a chunk of memory for use as the heap at startup, and then offer processes memory from those pages when requested, or does it actually allocate the heap memory as processes request it? I'd think for optimization purposes it'd have a pre-allocated pool to use, and increase it as needed, though I can't find docs to prove this theory.

2. Unless I'm mistaken, Windows would need to keep process-specific heap memory (such as memory requested through malloc) in a page entirely dedicated to that process - since the VMM maps full pages, it wouldn't be possible to store Process A and Process B's malloc'd data on the same page. This would seem to mean it has several pages dedicated to globally-accessible heap data, which are then mapped into each process at the same VM address. Is this how it works?
0
 
jimstarAuthor Commented:
Ok, I'm getting there. Let's say I have two processes - Process A and Process B.

Sequence of Events:
1. Process A allocates pBuffer1 (8 bytes) using malloc.
2. Process A allocates pBuffer2 (8 bytes) using GlobalAlloc.
3. Process A allocates pBuffer3 (8 bytes) using malloc.

--> At this point, Process B should be able to access pBuffer2 in its address space, but it should not be able to access pBuffer1 or pBuffer3. To accomplish this, it would seem that pBuffer1 and pBuffer3 need to be on a different memory page than pBuffer2 - right? Since memory is mapped/protected by page, malloc data and GlobalAlloc data cannot share a page, otherwise malloc data could potentially be mapped into another process when the GlobalAlloc data is mapped, if they happen to fall on the same page.
0
 
jkrConnect With a Mentor Commented:
As a simple image - yes. However, B can still gain access using 'ReadProcessMemory()' if A communicates the address. And 'GlobalAlloc()' does not allow to allocate shared pages. BTW, at some point, any allocators will somehow end up on Win32 API calls.
0
All Courses

From novice to tech pro — start learning today.