detect heap corruption BEFORE garbage collect

Hello all,
I am using cdb/windbg to try to force a break when heap corruption occurs by pinvoke into ReadFile.  The debugger does not see the access violation until after GC.Collect, which is **too late** for me. Prior to running my program, i run "gflags -p /enable testheap.exe /unaligned"  The effect seems useless.  I wrote this little test program to apply what I find to debugging a larger commercial program that is having heap corruption issues.

I have also tried DebugDiag with Application Verifier and MDA callbackOnCollectedDelegate without success.  Please help, as I have been struggling with this issue for a long time.

namespace TestHeap
{
  public partial class Form1 : Form
  {
    [DllImport("kernel32.dll", SetLastError = true)]
    static extern SafeFileHandle CreateFile(string lpFileName, uint dwDesiredAccess,
      uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition,
      uint dwFlagsAndAttributes, IntPtr hTemplateFile);
    
    [DllImport("kernel32.dll", SetLastError = true)]
    static extern bool ReadFile(SafeFileHandle hFile, [Out] byte[] lpBuffer,
       uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, IntPtr lpOverlapped);
    
    string fileName = "testHeap.txt";
    const uint GENERIC_READ = 0x80000000;
    const uint OPEN_EXISTING = 3;
    SafeFileHandle sh;
    byte[] chBuf = new byte[8];

    public Form1()
    {
      InitializeComponent();
    }

    private void testBtn_Click(object sender, EventArgs e)
    {
      bool nStat;
      uint bytesToRead = 1025;
      uint bytesRead = 0;

      if (!(nStat = ReadFile( sh, chBuf, bytesToRead, out bytesRead, IntPtr.Zero)))
        Debug.Print("testBtn_Click error in ReadFile, nStat = {0}", nStat);
      MessageBox.Show(string.Format("After ReadFile, bytesToRead = {0},\n bytes read = {1}", bytesToRead, bytesRead));
      GC.Collect();
      MessageBox.Show("testBtn_Click end, after GC.Collect");
    }

    private void Form1_Load(object sender, EventArgs e)
    {
      sh = CreateFile(fileName, GENERIC_READ, 0, IntPtr.Zero, OPEN_EXISTING, 0, IntPtr.Zero);
    }
  }
}

Open in new window

Tech_DrAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

SAMIR BHOGAYTAFreelancer and IT ConsultantCommented:
byte[] chBuf = new byte[8];

IntPtr chBuf = Marshal.AllocHGlobal(8);

[DllImport("kernel32.dll", SetLastError = true)]
static extern bool ReadFile(SafeFileHandle hFile, [Out] IntPtr lpBuffer,
uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, IntPtr lpOverlapped);
0
Tech_DrAuthor Commented:
samir:  Your answer appears to be a cut-and-paste from my post on stackoverflow.com.
0
Tech_DrAuthor Commented:
Several months ago I solved my large project problem without using the techniques described in the original post. It seems that the corrupt heap state in native code can't communicate with managed code in a timely manner, at least not with the tools described above. I am updating this with the hope that it may help save someone a lot of time.

The problem was found within my large project in a totally separate, unsuspected area: USB communication calls. Prior testing did not show any problems in this area. Nevertheless, I went through every single pinvoke call and substituted each call with an updated (third party) library call when possible. Also, I eliminated all "unsafe" pointer usage with alternatives.  Being that I had struggled with this for months, two or three days of inspecting all pinvoke calls was worth the effort.

In Visual Studio 2010 I was able to verify that heap corruption no longer occured in my updated code by using the debugging extension sos.dll, and periodically making the !verifyheap call manually by hand.  All's well with the program now - no crashes, no heap corruption.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Tech_DrAuthor Commented:
No other suggestion worked. This did.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
.NET Programming

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.