Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 493
  • Last Modified:

Memory error in a Windows service application (Win32 project) running on a Windows 2003 x64 Ed OS

Problem : fault error on a Win32 service application running on a W 2003 X64 OS

Environment:
Server: DELL PowerEdge 2850,Processor Intel EM64T 3GHz, with 13,3 RAM physical memory available
OS: Windows 2003 Standard x64 Edition
The server is dedicated to our software
Our software:
It is a search engine for Intranets, made of a classical dll (NoeBase.dll) running as a Windows service under the NoeService.exe application.
Both programs have been developed on Visual Studio 6 as Win32 projects which have worked well on 32bit W2003 with 2 GB or RAM, till they ran on a memory problem when indexing more than a certain amount of documents (say 200,000).
We decided to port it to a WIN64 project under Visual Studio 2005, to run them on the above DELL server.
Since we had a tight schedule, we chose to take a first step by recompiling both programs under Visual Studio 2005 as Win32 projects, linking them with /LARGEADDRESSAWARE setting in order to be able to use up to 4 GB of RAM.
NoeBase is located on C:\Windows\SysWOW64
NoeBase starts by allocating a heap of 128 MB, and reallocates the heap with chunks of more memory when needed.

The problem:
The NoeService application works well but runs into an error, as follows (DrWatson message)
"The application, NoeService.exe, generated an application error
The exception generated was c0000005 at address 7D611952 (ntdll!ExpInterlockedPopEntrySListFault)"
This error terminates the service.

From our app log file, the problem occurred in our function which reallocates the heap by calling HeapReAlloc
Pointer ResizeMem (Pointer AddrLpMem, indeX size)
{
      return deRef(AddrLpMem) = HeapReAlloc(_g_MemHeap, HEAP_GENERATE_EXCEPTIONS,
                              deRef(AddrLpMem),(DWORD)size);

}//ResizeMem

_g_MemHeap is a global handle obtained by call to GetProcessHeap()
Pointer is a void * type, and indeX an unsigned long type.
AddLpMem is the pointer to the block to reallocate.
deRef is a macro: #define deRef(P)      (*(Pointer *)(P))

The problem occurs when the Commit Charge is about 1,4 GB (of which approximately 900 MB used by NoeService).

Questions : How to prevent the error, either at the code level, or at the system level ?

This is a very important problem for us, and I am willing to give a high level of points.
0
Daniel Verney
Asked:
Daniel Verney
  • 4
  • 3
1 Solution
 
ZoppoCommented:
Hi verdan,

it could be that simply the process runs out of memory: A 32-bit process can't ever use more than 3GB of memory address range, no matter if it runs on 64-bit Windows or on 32-bit Windows-Server with '/3GB' switch.

Now imagine your service uses already about 1.4 GB - then the 'HeapReAlloc' may fail since the realloc in case of increasing size maybe can't be done by just reserving new memory to the end of the existing memory block - in this case a new memory block of the needed size is allocated, the data from the old memory block is copied to the new one, then the old one is released.

So, with memory-size of about 1.4 GB in this case the application needs nearly 3 GB to realloc the memory.

If you have to handle such amout of data I guess there are only two possibilities:

1. Compile it as a 64-bit application. This should solve the problem, at least as long as you don't allocate much more memory than the physical RAM of the machine.
2. Instead of reallocating the memory you could write the data to the disk, release the memory, allocate new memory and load the data from the disk - this will of course be a matter of performance.

Hope that helps,

ZOPPO
0
 
Daniel VerneyConsultantAuthor Commented:

Hi Zoppo,
If I understand your explanation, the realloc process needs more than twice the initial size, and I'had guessed that was something like that
I'm aware that solution 1 is the best, it's just a question of the time needed to do it
Solution 2 is actually possible with a little programming
However, I thought that with the /LARGEADDRESSAWARE setting in linking, it was possible to use 4 GB of memory for a Win32 app running on a Windows 64bit OS which has RAM enough (16 Gb) - at least it's what Microsoft says in its doc: do you have some hints about that question < this could give us some more time to implement solution 1>
(sorry my french keyboard does not translate well in this text box>>>) <<
Thanks
Verdan
0
 
ZoppoCommented:
Hi,

well, realloc either needs the size of the original memorey block (if new size is smaller), the size of the new memory block (if new size is larger but after the original block is enough free memory to extend it to the new size) or the sum of both sides if it needs to create o copy of the original memory block.

> it was possible to use 4 GB of memory for a Win32
That's not right, a 32-bit application cannot use more than 3GB, the rest is reserved for other data like program code, loaded DLLs and I guess some more.

ZOPPO
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
Daniel VerneyConsultantAuthor Commented:

Hi,
Thanks Zoppo,
But what about the /LARGEADDRESSAWARE setting ?
Microsoft explicitly says that it allows Win32 apps running on 64bits OS to use up to 4GB, if physically available, see
http://support.microsoft.com/default.aspx?scid=889654
Our server has 16 GB of physical RAM
Maybe I missed something ?

VERDAN
0
 
ZoppoCommented:
sorry, I was wrong - on 32-bin windows a process can never address more than 3GB, but there all DLLs, code a.s.o. even resides in this 3GB. In 64-bit windows for all together 4GB are available - but this still can be a problem when reallocating huge memory blocks - in fact to allocate a 1,5 GB block there has to be at least 1,5 GB free (unfragmented) memory. This can even fail with 4GB adress space ...

ZOPPO
0
 
Daniel VerneyConsultantAuthor Commented:
I wanted to accept all the comments that you sent me, but the Experts Exchange interface is not clear at all about accepting solutions (multiple?). What can I do ?
0
 
ZoppoCommented:
Hi,

you did it allright - there's no need to accept multiple comments from one expert - just if you have comments from multiple experts and find more than one helpful then you can accept one as answer and add a number of other comments as 'assisted'.

Have a nice day,

best regards,

ZOPPO
0

Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

  • 4
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now