Link to home
Start Free TrialLog in
Avatar of ironcladsecure
ironcladsecureFlag for United States of America

asked on

Increasing process memory limit

I'm trying to increase the amount of memory a process can use (we're running MySQL cluster on a machine with 4GB of memory; FreeBSD 6.0). I added the following values to /boot/loader.conf which worked:
kern.maxdsiz="1073741824" # 1GB
kern.dfldsiz="1073741824" # 1GB
kern.maxssiz="134217728" # 128MB

However, when I increased the kern.maxdsiz and kern.dfldsiz to 3GB, the system crashes at startup and says it was unable to dump.

Why does this work for 1GB, but not for 3GB? The system has plenty of physical memory.

Many thanks!

Ben
Avatar of ironcladsecure
ironcladsecure
Flag of United States of America image

ASKER

For your reference, here's the exact problem sequence when booting up:

"Trying to mount root from ufs:/dev/da0s1a
init died (signal 0, exit 1)
panic: Going nowhere without my init!
Uptime: 10s
Cannot dump. Np dump device defined.
Automatic reboot in 15 seconds ..."
32bit or 64bit?
MySQL - ports build or private build?
Ports build.
32bit
First of all you want to boot system.
Boot installer go to command shell.
mount /dev/da0s1a /mnt

cp /mnt/boot/loader.conf /mnt/boot/loader.conf.old
grep -v ^kern /mnt/boot/loader.conf.old > /mnt/boot/loader.conf

Now init will start.

You have changed default data size in kernel that makes init eat one gigabyte, rc yet another, subshells 3g each. it ends quickly since these processes run before swap space is allocated.

Then adjust only maximum limits of process size.

By default max datasize is half of RAM, you may override this to something that suits you better if you do not run other apps.

I was able to get the system to boot last night by unsetting kern.maxdsiz last night. The maximum process size I could allocate is 2.5GB.

So where specifically is the other 1.5GB going and how do I change it? Your answer seems a bit vague to me. All that is necessary is to change the maximum process size so it can allocate more like 3.5GB instead of 2.5. I can't imagine the system needs more than 512MB to run extremely well.

Right now I have the following line in my loader.conf file:
kern.maxdsiz="2684354560" # 2.5GB
kern.maxssiz="268435456" #256MB

Thanks!
There is limited address space of 4G for everything in 32bit i386 compatioble system

Your 32bit mainboard may eat AGP APERTURE and USB hardware buffers at very end of 4G
so 3.9G remain
Roughly 10% is used for memory management purposes
so 3.5G stay
256M is eaten away by kernel
ref: http://www.freebsd.org/doc/faq/advanced.html#CHANGE-KERNEL-ADDRESS-SPACE
so 3.2G stay

Rest is allocated to programs and free space to disk buffers.

3G is likely maximum.

Quick answer - run amd64 if compatible.

There is no problem with amount of memory - your system (or processor in 32bit mode)  has virtual address space that matches physical memory.

To increase address space you may experiment with compiling PAE kernel, but be warned that some drivers are not present here ( in /sys/i386/conf examine file named PAE and make sure your system does not depend on devices that will stop working)
Look at dmesg output - it shows available memory after kernel address space.
If running 64-bit, I realize the 4gb addressable space limit is removed, but it seems that I would still have the same problem in my scenario, right? I've got a machine with 4GB RAM, and the max I can have a process use is 2.5GB without the machine crashing. I know the OS doesn't need 1.5GB to run. I've run FreeBSD on boxes with 256MB RAM just fine. So how do I work around this?
You will have those 10% of address space which will leave database more breathing space i.e from roughly 3G to rougly 3.5G.
"You will have those 10% of address space"

What's that supposed to mean?
ASKER CERTIFIED SOLUTION
Avatar of gheist
gheist
Flag of Belgium image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Why B grade?
I checked dmesg, which tells me that I have 3839M real memory and 3759M available memory. But that still doesn't tell me where the rest of the memory is being allocated. top says I have 3730M available after system boot (of course like you said, some of this is allocated for processes. I increased kern.maxdsiz to 2764M which works. But that leaves over 1GB unallocated (or at least, where is it allocated?)

Your answers were helpful, but still don't answer that key question. Thanks.
3839 remains after it maps out hardware buffers set by BIOS.
3759 remains when some is reserved for memory manager
So far about your RAM

The problem is that you have 4G of ram and 4G of address space for process which makes quite a mess.

4G for process is usually split 2:2 or 3:1 or whatever between malloc() and kernel. So you get something (allocated for malloc)-(used by program and libs) at your disposal.

Quickly looking I did not find respective header to make binary-incompatibile FreeBSD i386 system that accesses close to 4G of RAM per program.