linux process more than 3GB memory
I need to have more than 3GB usable memory in single process. I know i can
- go to 64bit
- use 4G/4G kernel(but then limit is on 4G)
- create large shared memory block, then access it with small window(mmap/munmap). Not whole memory is available at same time, but it's ok for me.
OK, I want to go the last solution. I heard that there some standard/library/kernel-ex
I tried to google it out - no results. Anyone knows any references?
- go to 64bit
- use 4G/4G kernel(but then limit is on 4G)
- create large shared memory block, then access it with small window(mmap/munmap). Not whole memory is available at same time, but it's ok for me.
OK, I want to go the last solution. I heard that there some standard/library/kernel-ex
I tried to google it out - no results. Anyone knows any references?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Yes, I know I can do it (I described in my Q)
But I wanted something like AWE API from Windows.
And sybase is mentioning it... http://www.sybase.com/detail?id=1030009
But I wanted something like AWE API from Windows.
And sybase is mentioning it... http://www.sybase.com/detail?id=1030009
ASKER
My previous answer was for Kdo.
KeremE: hugemem kernels can use 64G of memory.
BTW: anybody have some link to performance comparison of standard 3/1G kernel to 4/4G kernel?
KeremE: hugemem kernels can use 64G of memory.
BTW: anybody have some link to performance comparison of standard 3/1G kernel to 4/4G kernel?
Hi Raven,
Let's back up and identify your environment. Can you tell us what hardware / OS is involved here?
Kent
ASKER
32bit kernel, 32bit application(multithreaded)
Kdo: I haven't examided Your code, but even it's buggy I know it can be done this way.
Kdo: I haven't examided Your code, but even it's buggy I know it can be done this way.
Which distro?
ASKER
RHEL. I'm aware of hugemem kernel.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Or you can use SMP kerne_ which can address as much as 16 GB. with 4 GB process size like Hugemem kernel and user space drivers won't need to be hugemem special ones. You can use standard applications with SMP kernel . Furthermore RHEL does not support hugemem in RHEL5 and you need to upgrade your support contract to RHEL AS support for RedHAT ES support will only cover your system if it has 16 GB or less memory.
ASKER
Wait. Please read my Q again. I'm aware of 4G/4G(hugemem) and 64bit systems. For some reasons I don;t want port the application to 64bit, and 4G is really temporary solution(and possibly have severe performance impact).
Hi Raven,
We're kind of wandering around in the dark now. You can do your own mapping in a 1G/3G system, install a kernel that supports 4G or greater, or upgrade the to 64 bits (which seems to be your least preferred solution).
What more can we suggest?
Kent
Raven: You have 3 alternatives (except from porting your app to 64 bit)
- Hugemem: Supports up to 64 GB, there's a need for compileing some modules. No need to modify your code. Why do you think is a temporary solution? What makes it temporary ?
- SMP: The moust useful no need extra compiled modules. No nned to modify your code.
- Highmem : This is the one with the greatest performance impact since kernel boundaries are at limit pthere's a process penalty of nearly 6%
These are only alternatives. Other than that with a standard kernel yu can not go beyong this bloody 3GB limit.
- Hugemem: Supports up to 64 GB, there's a need for compileing some modules. No need to modify your code. Why do you think is a temporary solution? What makes it temporary ?
- SMP: The moust useful no need extra compiled modules. No nned to modify your code.
- Highmem : This is the one with the greatest performance impact since kernel boundaries are at limit pthere's a process penalty of nearly 6%
These are only alternatives. Other than that with a standard kernel yu can not go beyong this bloody 3GB limit.
ASKER
> What more can we suggest?
A library, API, or kernel support for the mapping. That was the question. Really.
KeremE:
- Hugemem: 64G for process? Not at all. Please read more.
- SMP? can You elaborate how SMP can give me more ram for process?
- Highmem? Again, can You elaborate how HM can give me more ram for process?
A library, API, or kernel support for the mapping. That was the question. Really.
KeremE:
- Hugemem: 64G for process? Not at all. Please read more.
- SMP? can You elaborate how SMP can give me more ram for process?
- Highmem? Again, can You elaborate how HM can give me more ram for process?
ASKER
BTW: there's even easier way than mmap/munmap: shmat/shmdt with many shm areas.
Raven:
Hugemem: I told "Supports up to 64 GB" of memory, of course. 4 GB per process memory (as we'd talked earlier). Your program will work with work without modification and it will be able to allocate more than 3 GB.
SMP: 4 GB per process / 16 GB max memory. Your program will work with work without modification and it will be able to allocate more than 3 GB.
Highmem: Uses PAE up to 64 GB of mem / 4gb per process but with penalty 6% slower.
In these solution you won't need any extra API or something since all memory addressing thing is addressed by the kernel. All you'd do is to call vmalloc/ioremap to alloate more tha 3GB and it will succeed automatically if you use one of these kernels. They are only around 5% slower than the normal counterpart. But this is quite tolarable.
Another alternative isto use 64 architecture. it has a 4GB per process limitation for x32 applications but 512GB !!! per process limit usable with vmalloc/ioremap for native x86_64 application.
Hugemem: I told "Supports up to 64 GB" of memory, of course. 4 GB per process memory (as we'd talked earlier). Your program will work with work without modification and it will be able to allocate more than 3 GB.
SMP: 4 GB per process / 16 GB max memory. Your program will work with work without modification and it will be able to allocate more than 3 GB.
Highmem: Uses PAE up to 64 GB of mem / 4gb per process but with penalty 6% slower.
In these solution you won't need any extra API or something since all memory addressing thing is addressed by the kernel. All you'd do is to call vmalloc/ioremap to alloate more tha 3GB and it will succeed automatically if you use one of these kernels. They are only around 5% slower than the normal counterpart. But this is quite tolarable.
Another alternative isto use 64 architecture. it has a 4GB per process limitation for x32 applications but 512GB !!! per process limit usable with vmalloc/ioremap for native x86_64 application.
BTW - As KDO told earlier what other alternatives you're expect? Using a different kernel (SMP for ex.) requiring no change to your code ins't simple enough?
ASKER
> Another alternative isto use 64 architecture. it has a 4GB per process limitation for x32 applications but 512GB !!!
Oh, it's much more than that - believe me, I was testing ;)
Oh, it's much more than that - believe me, I was testing ;)
ASKER
I already pointed out what I expect.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Of course the target solution is 64bit. But before I have to workaround it. And if You read the thread, You should notice that I know how to accomplish that.
My question was if maybe there's some library ready for that. Or maybe there's some kernel patch which helps with this task. As You noticed Windows has such thing.
My question was if maybe there's some library ready for that. Or maybe there's some kernel patch which helps with this task. As You noticed Windows has such thing.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
OK, no such thing under linux (like AWE). Pitty. Not what I expected though.
ASKER
I'm going to make performance measurement: /dev/shm/+mmap against shmat/shmdt.
That's what left to me, right?
That's what left to me, right?
Since it is a VM file I don't think the performance will be very bad unless you do so much of swappng due to lack of memory. Most sources say unser nowmal conditions the performance decrease will be around 4% to 6%. You'll need lots of memory considering even the descriptors and pointers required to handle this file will be around 200 MB :)
As for this is not what you expect :) as EE experts we know that "ok you can't do it" counts as a valid answer in EE :)
Good luck
As for this is not what you expect :) as EE experts we know that "ok you can't do it" counts as a valid answer in EE :)
Good luck
First of all you can't go with a traditional 32-bit kernel. With a 32-bit kernell all you have is a 4GB address space and ind it is allocated as 3 GB for all processes and 1 GB kernel space. If you want to go beyond this you'll need one of these hugemem kernels which can address up to 16 GB with a 4 GB per process limit. Another way jou can shose to find some highmem kernels but they are modified by some people and d not a standard like a hugemem kernel. I'd suggest you to go for hugemem kernel to begin with.