• C

Huge memory requirments

Hi.

I'm writting a C program which uses up piles of memory. Currently it fails to allocate the memory I need (calling calloc returns with a null pointer). The thing is I think I *have* the actual memory requested. I use a Win2000 dual 1MHz station with 2Gig physical RAM and adaquate HD space. I'm writting in VC++ ver6 prof. Edition.

Here is my question: Can anyone suggest what I can do to make sure the program actually takes advantage of the full memory available? Again, although I use up a lot of mem, I am pretty sure there is still much left available which the programm is not using. Can this be true? If so, what should I do?

Frost
Frost_ByteAsked:
Who is Participating?
 
jonninCommented:
Win NT family suffers (or did) sometimes from memory fragmentation. I.E you have 1 gig of free ram, but (for example) its in alternating pages so you can only allocate 64k or something at a time. This is dated, I don't know if its still an issue, anyone?

Make sure the program does not somewhere allocate and deallocate a little memory thousands of times, which can cause this issue. Alternately, download and run a freeware program called rambooster, I think this may help.  The best fix is not to use dynamic memory anywhere you don't have to (won't fit on the stack, or other reasons like run time determined size, etc).  

If you truely do run out of memory, the "beyone dual processor" versions of windows (2k and/or xp?) can have more than 2gb memory, somewhat more expensive).

0
 
nebekerCommented:
For Windows 2000, I believe each process is limited to 2GB of memory.  How much memory are you trying to allocate?  Could you post a few lines of code that shows where the failed allocation is happening?

Also, what do you mean "make sure the program takes advantage of the full memory available" ???   Do you want to query the OS to find out how much memory is installed, and allocate a fixed percentage of it?
0
 
garbouaCommented:
well, I don't know much about w2k, but if you need more memory then you have, or could obtain then you can use pointer all over and then allocate and delete as needed.

if nebeke is correct, then you have a problem.  2bg of memory is a whole LOT of memory.  unless you are processing a very long segment of a graphical input, and I mean hours worth of input, You have a memory leak in your program.
your source code might help
0
How do you know if your security is working?

Protecting your business doesn’t have to mean sifting through endless alerts and notifications. With WatchGuard Total Security Suite, you can feel confident that your business is secure, meaning you can get back to the things that have been sitting on your to-do list.

 
John Mc HaleForensic Computer Examiner, Analyst/Programmer & Database ArchitectCommented:
Have you considered writing the program in C++ and using the operator new to allocate memory. This may be more efficient that the older calloc, realloc etc. library functions.
0
 
nebekerCommented:
fredthered -

There are no efficiencies gained by using "new" vs. malloc, unless you intend on writing your own new handler....
0
 
Frost_ByteAuthor Commented:
I'm an old timer, used to coding for DOS n'such. So my initial guess is that something is chocking me other than the actuall physical memory size. The OS, the compiler etc. I want to make sure that is not the case (say by asking you guys).

Posting code will be a problem - its a pretty large program, that has been "evolving" some time now, worked on by several people in the academy (you get the picture).

I don't have the exact numbers (how much mem exactly I'm trying to allocate) but I know its way less than what I have. This since calloc fails when I try to allocate memory to copy a certain array. The new array is as large as the existing one. Task manager tells me I still have way memory left, but it still fails.

Anything?
0
 
makerpCommented:
is it a dos program or a win32 application
0
 
Frost_ByteAuthor Commented:
Its a console program. Only includes stuff like stdio and stdlib, no windows.h etc.
0
 
makerpCommented:
what are you compiling it with?
0
 
Frost_ByteAuthor Commented:
VC++ ver 6. Everything is pretty much default in the settings.
0
 
makerpCommented:
strange... can you reproduce this behaviour with a simple console application that just loops and allocates memory..
0
 
oleberCommented:
Can't you work arround this problem.
Cann't you send some information for hard-disk?

If nebeker is correct, did you see how many Virtual Memory you are using?


0
 
nebekerCommented:
Just by chance, are you using a signed integer to hold the amount of memory being requested?  Depending on what type of variable (int, short int, etc.) you're using, and how much memory you're asking for, you might have overflowed the capacity of that variable -- which would mean you're asking for a "negative" amount of memory :)
0
 
Frost_ByteAuthor Commented:
nebeker : interesting idea, but no. I allocate the same amount of memory twice (this happens when I make a copy of a large _existing_ array). The first time it works, only fails on the second try. That and I calloc an array of length smaller than 10mil of large structs. The command is ...calloc(N * sizeof(BigStruct)). No signed\unsigned issues, as far as I can tell.

oleber : Have around 5gig swap file. I think that should be enough for my needs.

makerp : I'll have to try that but it'll take me time. Will get back to you on that today-tomorrow.

I understand that there is no general known issue with memory limitations, right? There is no hidden switch labeled "let me use everything you got" that I haven't been pushing?
0
 
weiccoCommented:
Have tried to call GetLastError / FormatMessage after calloc returns NULL, what does it say?

Hmm... I had wild idea. Try to allocate memory with CreateFileMapping.
0
 
CSLIUCommented:
From 386 to IA32, processors are 32-bit.
The maximum physical addressing space is 4GB in 386 protected mode.

http://support.microsoft.com/support/kb/articles/Q268/3/63.asp

http://support.microsoft.com/support/kb/articles/Q283/0/37.ASP


C code:
/* an example */
#include <stdlib.h>
#include <stdio.h>
int main()
{
 void  *largeP;
 unsigned int uSize;   /* unsigned int is 4bytes(32-bit) in VC++6.0 */

 uSize = 0x80000000 - 1;   /* 2G-1 */

 largeP = malloc( uSize );
 if( largeP == NULL )
 {
   printf("Ok");
   free( largeP );
 }
 else
  printf("Falied");

 return 0;
}
0
 
weiccoCommented:
"The best fix is not to use dynamic memory anywhere you don't have to (won't fit on the stack, or other reasons like run time determined size, etc)."

Doesn't malloc use heap not stack?
0
 
Frost_ByteAuthor Commented:
Well I downloaded some mem-checking utils, not rambooster, but FreeRam XP pro (also free). I still haven't solved the problem but I now know that I have a very fragmented memory which I guess is causing all this. I am, in fact, doing tons of allocating, freeing of small mem fragments. Looks like I have no way of avoiding actual recoding.

Thanks all,
Frost
0
 
weiccoCommented:
Little comment... If you need to allocate & realloca & free lots of small memory fragments I suggest you write your own memory manager.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.