Memory > 4GB RH AS3.0

I have just installed Red Hat AS3.0 (32 bit) on a 4CPU box (AMD Opteron) with 8GB of memory and yet I only see 4GB of memory in /proc/meminfo. I am running the SMP kernal and my understanding that it was good for 16GB of memory (uname -a yields: 2.4.21-27.0.2.ELsmp #1 SMP Wed Jan 12 23:24:48 EST 2005 i686 athlon i386 GNU/Linux). I added the mem=8000MB to /boot/grub/brub.conf and rebooted to no avail:
title Red Hat Enterprise Linux AS (2.4.21-27.0.2.ELsmp)
        root (hd0,0)
        kernel /vmlinuz-2.4.21-27.0.2.ELsmp ro root=LABEL=/
        initrd /initrd-2.4.21-27.0.2.ELsmp.img mem=8000MB
title Red Hat Enterprise Linux AS (2.4.21-27.0.2.EL)
        root (hd0,0)
        kernel /vmlinuz-2.4.21-27.0.2.EL ro root=LABEL=/

Do I need a different kernel or just set something to get all 8GB??

Thanks for any help.
        initrd /initrd-2.4.21-27.0.2.EL.img
Who is Participating?
TaleoConnect With a Mentor Commented:

We isolated the problem and have a fix.

When installing RHEL 3.0, it detects the Opteron CPU and installs the Athlon version of the kernel-smp instead
of the i686 version that it uses for Intel CPUs.  But, for some reason, the Athlon smp kernel does not have the
PAE and highpte options enabled, which means it is limited to 4G... but the i686 version of kernel-smp does
have these options, and works just fine (and is supported) on Opteron.  See the following bug at RedHat:

So, we have to tweak our install kit to install the i686 kernel-smp and it works great, all 8GB RAM available,
and all our current kernel modules work without modifications just like on our Xeon systems, and we don't
have to use the 4/4 memory split that the hugemem kernel imposes.  Thats the compatibility we were looking

That's it!


> uname -a yields: 2.4.21-27.0.2.ELsmp #1 SMP Wed Jan 12 23:24:48 EST 2005 i686 athlon i386 GNU/Linux
Opteron is 64bit CPU. So you should download RHEL AS 3.0 (x86_64, 64bit) CDs and install this version.
# uname -m
With 64bit kernel, you can use up to 2TB memory. Also, your single swap partition could be 16GB or more.
Also, Opteron is backward compatible to 32bit softwares, you just need to install those 32bit libraries/softwares, too.
When you instatll RHEL, select "custom" and check
"Comaptibility Arch Development Support".

Otherwise, you need to install kernel-hugemem-2.4.21-27.0.4 (2.4.21-27.0.4 just released a few days ago)
to use the memory over 4GB.
richl796Author Commented:
64 bit isn't an option right now because I'll be putting Oracle on the machine and they only certify with 32 bit OS on the Opteron CPU. I've seen the hugemem kernel before but I thought it went like this for memory sizes:
<4GB regular kernel
>4GB <16GB smp kernel
>16GB hugemem

Maybe I'm just confused as to which kernel is which?
Cloud Class® Course: Microsoft Windows 7 Basic

This introductory course to Windows 7 environment will teach you about working with the Windows operating system. You will learn about basic functions including start menu; the desktop; managing files, folders, and libraries.

SMP (Symmetric Multiple Processor) is for multiple CPUs or Hyperthread enable.
Other than multiple CPU, smp is pretty much the same as regular single CPU kernel with regarding to memory size.
hugemem may have some overhead due to handling large memory, but the overall is quiute good.
Redhat's kernel change the specification from time to time.
So try hugemem first.
richl796Author Commented:
 I understand what you're saying. I have a 4 CPU box so that's why I'm using the SMP kernel. It clearly states on RHN that th hugemem kernel is for systems with over 16GB of memory. I have even seen references to people using the regular smp kernel with 8GB of memory in Redhat's newsgroups. The think is no one ever says how they got the machine to see the 8GB. I can't believe that I have to compile the kernel by hand or use the hugemem kernel wich is overkill to see 8GB of memory. It seems that that's a pretty standard config now days and should work out of the box. Of course I could be wrong...;->
RedHat change the specification quite often.
Before, hugemem is for 4GB or above, now, new version of kernel, it support from 16GB or above.
richl796Author Commented:
So that leads me back to my original situation. I only have 8GB of memory so I don't need hugemem. Yet how can I get the regular SMP kernel to see the 8GB's that it is supposed to?
No, what I say is RedHat claim that. But just try hugemem first.
You can install multiple kernel an choose any kernel to boot.
If you can not see 8GB memory in hugemem kernel, then there is something wrong with others such as BIOS setting.
richl796Author Commented:
I know it's not the machine because if I install the 64 bit kernel I get the 8GB to show up. I'll try the 32 bit hugemem kernel though and report back.
We have the same problem.

We recently acquired HP DL145 servers with Dual Opterons and 8GB RAM (8x1GB).
We tried our standard kickstart with RHEL 3.0 32-bit that works great with Dual Xeons
that have 8GB of RAM, both the pre-EM64T Xeons as well as the newer EM64T Xeons.
But unless we use the hugemem kernel, we only see 4GB RAM... we are not ready yet
to bring our apps to 64-bit, and I don't want to use the hugemem 32-bit kernel
either.  So this really sucks.  We tried using mem=8GB in the kernel parameters, but
no dice there.

When we tried the hugemem kernel, all 8GB were visible... so the box works ok.
I wonder why the standard kernel can't see all the RAM... weird stuff.

> I wonder why the standard kernel can't see all the RAM... weird stuff.
Becasue the Linux kernel 2.4.x for 32bit architecture can only see 2GB (4GB with "double long").
It is the limitation of 32bit architecture.
RedHat developement team change the memory management to allow OS to see and use the memory
beyond the 4GB. It is implemented as "hugemem" kernel.
As I noticed, before, without hugemem kernel, you can only see 4GB.
But recent RedHat release claims that the regular kernel support up to 16GB (??).

Hugemem did have some overhead but as RedHat' announcement, hugemem will improve the
memory utilization if you system have a lot of memory.

As my experience, I don't see the negative impact to use hugemem even though some our webservers have
only 2GB memory.

Besides, the Opteron or EMT64 CPU, you can still install 64bit OS with 32bit compatible packages as my first post.
There is no hugmem for x86_64 or ia32e kernel.
I just setup 4 EMT64 OS 0ne month ago for testing. So far no 32bit application has issue yet.

richl796Author Commented:
After talking to Red Hat support they told me I have to use the 32-bit hugemem kernel to see my 8GBs. They need to update their documentation to explain this better. Going to 64 bit is not an option for a lot of people. Oracle only certifies on 32-bit Red Hat on the Opteron processor.
> Red Hat support they told me I have to use the 32-bit hugemem kernel to see my 8GBs.
That's my experience and what I expect.
Why don't you just try the hugemem to see any performance issue?
That's a good link.
It did mention about enabling HIGHMEM64G would break all 3rd party kernel modules.
You might want to take this into consideration.

Actually, we aren't recompiling anything - we are simply using the stock provided i686 kernel-smp.
So no broken modules here :-)  We simply use the same software we are using on our Xeon servers.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.