?
Solved

detect heap corruption BEFORE garbage collect

Posted on 2011-10-30
4
Medium Priority
?
412 Views
Last Modified: 2013-12-16
Hello all,
I am using cdb/windbg to try to force a break when heap corruption occurs by pinvoke into ReadFile.  The debugger does not see the access violation until after GC.Collect, which is **too late** for me. Prior to running my program, i run "gflags -p /enable testheap.exe /unaligned"  The effect seems useless.  I wrote this little test program to apply what I find to debugging a larger commercial program that is having heap corruption issues.

I have also tried DebugDiag with Application Verifier and MDA callbackOnCollectedDelegate without success.  Please help, as I have been struggling with this issue for a long time.

namespace TestHeap
{
  public partial class Form1 : Form
  {
    [DllImport("kernel32.dll", SetLastError = true)]
    static extern SafeFileHandle CreateFile(string lpFileName, uint dwDesiredAccess,
      uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition,
      uint dwFlagsAndAttributes, IntPtr hTemplateFile);
    
    [DllImport("kernel32.dll", SetLastError = true)]
    static extern bool ReadFile(SafeFileHandle hFile, [Out] byte[] lpBuffer,
       uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, IntPtr lpOverlapped);
    
    string fileName = "testHeap.txt";
    const uint GENERIC_READ = 0x80000000;
    const uint OPEN_EXISTING = 3;
    SafeFileHandle sh;
    byte[] chBuf = new byte[8];

    public Form1()
    {
      InitializeComponent();
    }

    private void testBtn_Click(object sender, EventArgs e)
    {
      bool nStat;
      uint bytesToRead = 1025;
      uint bytesRead = 0;

      if (!(nStat = ReadFile( sh, chBuf, bytesToRead, out bytesRead, IntPtr.Zero)))
        Debug.Print("testBtn_Click error in ReadFile, nStat = {0}", nStat);
      MessageBox.Show(string.Format("After ReadFile, bytesToRead = {0},\n bytes read = {1}", bytesToRead, bytesRead));
      GC.Collect();
      MessageBox.Show("testBtn_Click end, after GC.Collect");
    }

    private void Form1_Load(object sender, EventArgs e)
    {
      sh = CreateFile(fileName, GENERIC_READ, 0, IntPtr.Zero, OPEN_EXISTING, 0, IntPtr.Zero);
    }
  }
}

Open in new window

0
Comment
Question by:Tech_Dr
  • 3
4 Comments
 
LVL 11

Expert Comment

by:SAMIR BHOGAYTA
ID: 37054863
byte[] chBuf = new byte[8];

IntPtr chBuf = Marshal.AllocHGlobal(8);

[DllImport("kernel32.dll", SetLastError = true)]
static extern bool ReadFile(SafeFileHandle hFile, [Out] IntPtr lpBuffer,
uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, IntPtr lpOverlapped);
0
 

Author Comment

by:Tech_Dr
ID: 37083951
samir:  Your answer appears to be a cut-and-paste from my post on stackoverflow.com.
0
 

Accepted Solution

by:
Tech_Dr earned 0 total points
ID: 37584517
Several months ago I solved my large project problem without using the techniques described in the original post. It seems that the corrupt heap state in native code can't communicate with managed code in a timely manner, at least not with the tools described above. I am updating this with the hope that it may help save someone a lot of time.

The problem was found within my large project in a totally separate, unsuspected area: USB communication calls. Prior testing did not show any problems in this area. Nevertheless, I went through every single pinvoke call and substituted each call with an updated (third party) library call when possible. Also, I eliminated all "unsafe" pointer usage with alternatives.  Being that I had struggled with this for months, two or three days of inspecting all pinvoke calls was worth the effort.

In Visual Studio 2010 I was able to verify that heap corruption no longer occured in my updated code by using the debugging extension sos.dll, and periodically making the !verifyheap call manually by hand.  All's well with the program now - no crashes, no heap corruption.
0
 

Author Closing Comment

by:Tech_Dr
ID: 37607568
No other suggestion worked. This did.
0

Featured Post

NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
This article shows how to deploy dynamic backgrounds to computers depending on the aspect ratio of display
This video shows how to quickly and easily deploy an email signature for all users in Office 365 and prevent it from being added to replies and forwards. (the resulting signature is applied on the server level in Exchange Online) The email signat…
The Relationships Diagram is a good way to get an overall view of what a database is keeping track of. It is also where relationships are defined. A relationship specifies how two tables connect to each other. As you build tables in Microsoft Ac…
Suggested Courses
Course of the Month12 days, 17 hours left to enroll

579 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question