Increasing process memory limit

I'm trying to increase the amount of memory a process can use (we're running MySQL cluster on a machine with 4GB of memory; FreeBSD 6.0). I added the following values to /boot/loader.conf which worked:
kern.maxdsiz="1073741824" # 1GB
kern.dfldsiz="1073741824" # 1GB
kern.maxssiz="134217728" # 128MB

However, when I increased the kern.maxdsiz and kern.dfldsiz to 3GB, the system crashes at startup and says it was unable to dump.

Why does this work for 1GB, but not for 3GB? The system has plenty of physical memory.

Many thanks!

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ironcladsecureAuthor Commented:
For your reference, here's the exact problem sequence when booting up:

"Trying to mount root from ufs:/dev/da0s1a
init died (signal 0, exit 1)
panic: Going nowhere without my init!
Uptime: 10s
Cannot dump. Np dump device defined.
Automatic reboot in 15 seconds ..."
32bit or 64bit?
MySQL - ports build or private build?
ironcladsecureAuthor Commented:
Ports build.
OWASP: Forgery and Phishing

Learn the techniques to avoid forgery and phishing attacks and the types of attacks an application or network may face.

ironcladsecureAuthor Commented:
First of all you want to boot system.
Boot installer go to command shell.
mount /dev/da0s1a /mnt

cp /mnt/boot/loader.conf /mnt/boot/loader.conf.old
grep -v ^kern /mnt/boot/loader.conf.old > /mnt/boot/loader.conf

Now init will start.

You have changed default data size in kernel that makes init eat one gigabyte, rc yet another, subshells 3g each. it ends quickly since these processes run before swap space is allocated.

Then adjust only maximum limits of process size.

By default max datasize is half of RAM, you may override this to something that suits you better if you do not run other apps.

ironcladsecureAuthor Commented:
I was able to get the system to boot last night by unsetting kern.maxdsiz last night. The maximum process size I could allocate is 2.5GB.

So where specifically is the other 1.5GB going and how do I change it? Your answer seems a bit vague to me. All that is necessary is to change the maximum process size so it can allocate more like 3.5GB instead of 2.5. I can't imagine the system needs more than 512MB to run extremely well.

Right now I have the following line in my loader.conf file:
kern.maxdsiz="2684354560" # 2.5GB
kern.maxssiz="268435456" #256MB

There is limited address space of 4G for everything in 32bit i386 compatioble system

Your 32bit mainboard may eat AGP APERTURE and USB hardware buffers at very end of 4G
so 3.9G remain
Roughly 10% is used for memory management purposes
so 3.5G stay
256M is eaten away by kernel
so 3.2G stay

Rest is allocated to programs and free space to disk buffers.

3G is likely maximum.

Quick answer - run amd64 if compatible.

There is no problem with amount of memory - your system (or processor in 32bit mode)  has virtual address space that matches physical memory.

To increase address space you may experiment with compiling PAE kernel, but be warned that some drivers are not present here ( in /sys/i386/conf examine file named PAE and make sure your system does not depend on devices that will stop working)
Look at dmesg output - it shows available memory after kernel address space.
ironcladsecureAuthor Commented:
If running 64-bit, I realize the 4gb addressable space limit is removed, but it seems that I would still have the same problem in my scenario, right? I've got a machine with 4GB RAM, and the max I can have a process use is 2.5GB without the machine crashing. I know the OS doesn't need 1.5GB to run. I've run FreeBSD on boxes with 256MB RAM just fine. So how do I work around this?
You will have those 10% of address space which will leave database more breathing space i.e from roughly 3G to rougly 3.5G.
ironcladsecureAuthor Commented:
"You will have those 10% of address space"

What's that supposed to mean?
When you look at dmesg you see it sets aside some RAM.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Why B grade?
ironcladsecureAuthor Commented:
I checked dmesg, which tells me that I have 3839M real memory and 3759M available memory. But that still doesn't tell me where the rest of the memory is being allocated. top says I have 3730M available after system boot (of course like you said, some of this is allocated for processes. I increased kern.maxdsiz to 2764M which works. But that leaves over 1GB unallocated (or at least, where is it allocated?)

Your answers were helpful, but still don't answer that key question. Thanks.
3839 remains after it maps out hardware buffers set by BIOS.
3759 remains when some is reserved for memory manager
So far about your RAM

The problem is that you have 4G of ram and 4G of address space for process which makes quite a mess.

4G for process is usually split 2:2 or 3:1 or whatever between malloc() and kernel. So you get something (allocated for malloc)-(used by program and libs) at your disposal.

Quickly looking I did not find respective header to make binary-incompatibile FreeBSD i386 system that accesses close to 4G of RAM per program.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.