Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 332
  • Last Modified:

Huge memory requirments

Hi.

I'm writting a C program which uses up piles of memory. Currently it fails to allocate the memory I need (calling calloc returns with a null pointer). The thing is I think I *have* the actual memory requested. I use a Win2000 dual 1MHz station with 2Gig physical RAM and adaquate HD space. I'm writting in VC++ ver6 prof. Edition.

Here is my question: Can anyone suggest what I can do to make sure the program actually takes advantage of the full memory available? Again, although I use up a lot of mem, I am pretty sure there is still much left available which the programm is not using. Can this be true? If so, what should I do?

Frost
0
Frost_Byte
Asked:
Frost_Byte
  • 5
  • 3
  • 3
  • +6
1 Solution
 
nebekerCommented:
For Windows 2000, I believe each process is limited to 2GB of memory.  How much memory are you trying to allocate?  Could you post a few lines of code that shows where the failed allocation is happening?

Also, what do you mean "make sure the program takes advantage of the full memory available" ???   Do you want to query the OS to find out how much memory is installed, and allocate a fixed percentage of it?
0
 
garbouaCommented:
well, I don't know much about w2k, but if you need more memory then you have, or could obtain then you can use pointer all over and then allocate and delete as needed.

if nebeke is correct, then you have a problem.  2bg of memory is a whole LOT of memory.  unless you are processing a very long segment of a graphical input, and I mean hours worth of input, You have a memory leak in your program.
your source code might help
0
 
John Mc HaleForensic Computer Examiner, Analyst/Programmer & Database ArchitectCommented:
Have you considered writing the program in C++ and using the operator new to allocate memory. This may be more efficient that the older calloc, realloc etc. library functions.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
nebekerCommented:
fredthered -

There are no efficiencies gained by using "new" vs. malloc, unless you intend on writing your own new handler....
0
 
Frost_ByteAuthor Commented:
I'm an old timer, used to coding for DOS n'such. So my initial guess is that something is chocking me other than the actuall physical memory size. The OS, the compiler etc. I want to make sure that is not the case (say by asking you guys).

Posting code will be a problem - its a pretty large program, that has been "evolving" some time now, worked on by several people in the academy (you get the picture).

I don't have the exact numbers (how much mem exactly I'm trying to allocate) but I know its way less than what I have. This since calloc fails when I try to allocate memory to copy a certain array. The new array is as large as the existing one. Task manager tells me I still have way memory left, but it still fails.

Anything?
0
 
makerpCommented:
is it a dos program or a win32 application
0
 
Frost_ByteAuthor Commented:
Its a console program. Only includes stuff like stdio and stdlib, no windows.h etc.
0
 
makerpCommented:
what are you compiling it with?
0
 
Frost_ByteAuthor Commented:
VC++ ver 6. Everything is pretty much default in the settings.
0
 
makerpCommented:
strange... can you reproduce this behaviour with a simple console application that just loops and allocates memory..
0
 
oleberCommented:
Can't you work arround this problem.
Cann't you send some information for hard-disk?

If nebeker is correct, did you see how many Virtual Memory you are using?


0
 
nebekerCommented:
Just by chance, are you using a signed integer to hold the amount of memory being requested?  Depending on what type of variable (int, short int, etc.) you're using, and how much memory you're asking for, you might have overflowed the capacity of that variable -- which would mean you're asking for a "negative" amount of memory :)
0
 
Frost_ByteAuthor Commented:
nebeker : interesting idea, but no. I allocate the same amount of memory twice (this happens when I make a copy of a large _existing_ array). The first time it works, only fails on the second try. That and I calloc an array of length smaller than 10mil of large structs. The command is ...calloc(N * sizeof(BigStruct)). No signed\unsigned issues, as far as I can tell.

oleber : Have around 5gig swap file. I think that should be enough for my needs.

makerp : I'll have to try that but it'll take me time. Will get back to you on that today-tomorrow.

I understand that there is no general known issue with memory limitations, right? There is no hidden switch labeled "let me use everything you got" that I haven't been pushing?
0
 
weiccoCommented:
Have tried to call GetLastError / FormatMessage after calloc returns NULL, what does it say?

Hmm... I had wild idea. Try to allocate memory with CreateFileMapping.
0
 
jonninCommented:
Win NT family suffers (or did) sometimes from memory fragmentation. I.E you have 1 gig of free ram, but (for example) its in alternating pages so you can only allocate 64k or something at a time. This is dated, I don't know if its still an issue, anyone?

Make sure the program does not somewhere allocate and deallocate a little memory thousands of times, which can cause this issue. Alternately, download and run a freeware program called rambooster, I think this may help.  The best fix is not to use dynamic memory anywhere you don't have to (won't fit on the stack, or other reasons like run time determined size, etc).  

If you truely do run out of memory, the "beyone dual processor" versions of windows (2k and/or xp?) can have more than 2gb memory, somewhat more expensive).

0
 
CSLIUCommented:
From 386 to IA32, processors are 32-bit.
The maximum physical addressing space is 4GB in 386 protected mode.

http://support.microsoft.com/support/kb/articles/Q268/3/63.asp

http://support.microsoft.com/support/kb/articles/Q283/0/37.ASP


C code:
/* an example */
#include <stdlib.h>
#include <stdio.h>
int main()
{
 void  *largeP;
 unsigned int uSize;   /* unsigned int is 4bytes(32-bit) in VC++6.0 */

 uSize = 0x80000000 - 1;   /* 2G-1 */

 largeP = malloc( uSize );
 if( largeP == NULL )
 {
   printf("Ok");
   free( largeP );
 }
 else
  printf("Falied");

 return 0;
}
0
 
weiccoCommented:
"The best fix is not to use dynamic memory anywhere you don't have to (won't fit on the stack, or other reasons like run time determined size, etc)."

Doesn't malloc use heap not stack?
0
 
Frost_ByteAuthor Commented:
Well I downloaded some mem-checking utils, not rambooster, but FreeRam XP pro (also free). I still haven't solved the problem but I now know that I have a very fragmented memory which I guess is causing all this. I am, in fact, doing tons of allocating, freeing of small mem fragments. Looks like I have no way of avoiding actual recoding.

Thanks all,
Frost
0
 
weiccoCommented:
Little comment... If you need to allocate & realloca & free lots of small memory fragments I suggest you write your own memory manager.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 5
  • 3
  • 3
  • +6
Tackle projects and never again get stuck behind a technical roadblock.
Join Now