Link to home
Start Free TrialLog in
Avatar of DSOM
DSOMFlag for United States of America

asked on

Examining PE header with mapped file

I have a DLL that maps a file and another DLL that returns information about the mapped file.  The problem is that the function which examines the PE header faults after a few hundred files.  I have not been able to find a handle leak.  Here is the source code:

library ms0001;

{$E dll}
{$R *.res}

uses
  windows;

function tc_openfileview(name: pchar; size: cardinal): pointer; stdcall; export;
var hmap: thandle;
    fh: thandle;
begin
  fh:=createfile(name, GENERIC_READ, FILE_SHARE_READ, nil, OPEN_EXISTING, 0, 0);

  if (fh > 0) then
    begin
      hmap:=createfilemapping(fh, nil, PAGE_READONLY, 0, 0, 'TC_FILE_MAP');
      closehandle(fh);

      if (hmap > 0) then
        result:=mapviewoffile(hmap, FILE_MAP_READ, 0, 0, size)
         else
        result:=nil;
      closehandle(hmap);
    end
     else
    result:=nil;
end;

function tc_closefileview(map: pointer): boolean; stdcall; export;
begin
  result:=unmapviewoffile(map);
end;

exports tc_openfileview, tc_closefileview;

begin
end.


library ms0003;

{$E dll}
{$R *.res}

uses
  windows;

function CheckPEFile(AData:PByte):Boolean; stdcall; export;
//return True if AData points on valid image file
var
 LPNtHdr:PImageNtHeaders;
begin
 Result:=False;
 try
  if PImageDosHeader(AData)^.e_magic<>PWord(PChar('MZ'))^ then Exit;

  LPNtHdr:=Pointer(Cardinal(AData)+Cardinal(PImageDosHeader(AData)^._lfanew));

  if LPNtHdr^.Signature<>PCardinal(PChar('PE'))^ then Exit;       //seems to always fault here

  if LPNtHdr^.FileHeader.Machine<>IMAGE_FILE_MACHINE_I386 then Exit;
  if LPNtHdr^.OptionalHeader.Magic<>IMAGE_NT_OPTIONAL_HDR_MAGIC then Exit;
  Result:=True;
 except
 end;
end;

exports checkpefile;

begin
end.

The main program iterates all files on drive C:\ and passes them to openfileview(filename, 4096) and then uses the returned pointer to pass to CheckPEFile

Avatar of Member_2_248744
Member_2_248744
Flag of United States of America image

hello DSOM, First, I will say that I used to do Mem mapped files the way you do in your code above, as I would Open a File with CreateFile( ) and then use CreateFileMapping(  ) to get a map handle and then imediately close the file handle with CloseHandle( ), then I would get the map pointer with MapViewOfFile(  ) and imediately close the file map handle with CloseHandle(hmap), , and this would work most of the time, BUT I had some problems, sometimes Inconsistant, so now I DO NOT close the file handle or the map Handle untile AFTER I call UnMapViewOfFile(  ), so the sequence I use now is

UnMapViewOfFile(pMap);
CloseHandle(hMap);
CloseHandle(hFile);

you may can use a record with the three open values in it ? ?

BUT this may not solve your problem, your way seems to work most of the time. . .

OK Next, I think you may have memory reading problems with the pointer assignmemt in -
 if LPNtHdr^.Signature<>PCardinal(PChar('PE'))^ then Exit;

the   PCardinal(PChar('PE'))^   is a 4 Byte assignment  PCardinal^  of a 3 byte memory block PChar('PE')^ you may want to "Pad" the PChar mem block to make it 4 bytes  PChar('PEX') , it has a null for the last byte    OR change it to  PWord(PChar('PE'))^ for a 2 Byte read.
But I did not try this so I am not sure about it as a problem solve
Avatar of DSOM

ASKER

I tried both suggestions but it still faults in the same place.  It's very frustrating.  It's always an access violation on reading the address.  Perhaps the mapped file view needs to be locked somehow.  I'll have to look into that.
Avatar of AmigoJack
AmigoJack

unreasonable behaviour of major projects (or calculations) took me to the following: check project options - turn off code optimization and also turn off any compiler or syntax helping, turn on more strict rules. this might give you a few more compiler warnings or even errors, because standard settings are set for quite laidback programming. ok, this was under delphi 5, dont know which version youre using
Avatar of Wim ten Brink
Filehandles can be negative values but you check for 'if (fh > 0) then' meaning that you might be forgetting to close a few files.
Another problem could be that you're trying to examine a file that is smaller than the size you provided. There are a few such executables in the C:\Windows\System32 folder, which are basically some of the old MS-DOS commands like fastopen.exe and share.exe. Opening those files with a size of 4096 might of course lead to nasty problems.
negative? Cardinal = DWord = THandle = HWND are 4 bytes unsigned. but checking for > 0 is indeed incorrect. the checking should be

if fh<> INVALID_FILE_HANDLE

for 99% i swear that createfile() returns MAX_DWORD when an error occurs instead of 0. see win32.hlp for more
Avatar of DSOM

ASKER

I am not opening the file but mapping it and the 4096 is a maximum value not an absolute.  A zero-byte file gives a nil value and a 1 byte file gives you 1 byte.

I tried with if (fh <> INVALID_HANDLE_VALUE) then instead of > 0 and it didn't change anything.

I am using Borland Developer Studio 2006

All the project options are already set to to strict rules and no optimiztions.
ASKER CERTIFIED SOLUTION
Avatar of Russell Libby
Russell Libby
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Just as a suggestion, If your problem seems to be only with the file mapping, (not with opening the files with CreateFile), You can skip the mapping of the file, and just use your own Memory block, , I beleive the system will just create a system mem block and read the file comtents into it for a mam map fie, , You can just have the function   GetMem, or other memory allocation, and then just read the file into the mem block, then close the file handle. and do your PE analisis on your mem block, , go chance for map file to screw it up

That may speed things up, but the size of the data read into the memory block needs to be tracked, *period*. This is where the true problems lay:

For example:

>>  if PImageDosHeader(AData)^.e_magic<>PWord(PChar('MZ'))^ then Exit;
What if memory is only mapped to 1 byte?

>>  LPNtHdr:=Pointer(Cardinal(AData)+Cardinal(PImageDosHeader(AData)^._lfanew));
No check done to ensure the memory size >= size of TImageDosHeader. If the file just happens to start with MZ, then your in trouble. No validation done to ensure that the memory size is at least _lfanew bytes long + size of the nt header.

>>  if LPNtHdr^.Signature<>PCardinal(PChar('PE'))^ then Exit;      
Bad way to check memory, as the static PChar 'PE'#0 only occupies 3 bytes, but is typecast as ptr to a 4 byte struct

If a file starts with 'MZ' then the show is pretty much over because (a), the memory block wont be large enough to satisfy the dos header struct size, or (b) will be large enough, but god knows what the memory at _lfanew will translate into integer-wise.

Russell