• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 857
  • Last Modified:

CryptAPI Not Working in ActiveX DLL


I am trying to write an ActiveX DLL that makes a call to the CryptAcquireContext API.  Everything works perfectly when I cut and paste the code directly into an executable and run it on the same machine.

The ActiveX DLL is being called from an asp script on a web server and it is failing with code 87

The API call is:
lRet = CryptAcquireContext(hCtx, "", "", 1, 0)


PS: Please don't anybody give me a link to a class or dll that does encryption for me.  I specifically want to hash a string from within this DLL and not using massive code overhead that I don't need.  I'm only interested in why this isn't working and how to get around it (or in an easy to implement hash algorithm if anyone has one)
  • 2
  • 2
1 Solution
Error 87 is "The parameter is incorrect."

In looking over your parameters, I would have expected you to use vbNullString instead of "".  The 1 (PROV_RSA_FULL) looks fine, although I might explicitly qualify it with a typechar (i.e., 1&) just to make sure it was full size.  Ditto for the Flag parameter (0).  I can only assume that the hCtx is Dim'ed properly.

Other than that, you might post the Declare statement that you're using.  It's probably fine, since you say that it's working in an EXE.  Along those lines, since the Container and Provider parameters are pointers to strings, if I were in your shoes and it still wasn't working, I might alter the Declare to change them from String to Long and pass zeros (since it appears you're trying to pass NULL for those parameters).

Honestly, given the error message and the line of code, I'd really suspect the empty string as being the culprit.  If you can, give vbNullString a try and, if that doesn't work, post back here with your Declare.

And just to make sure: the error 87 is the value of lRet which is in the DLL, not in the ASP, right?

DrackonAuthor Commented:
I have tried the vbNullString and the qualifying as suggested and altering the Declare.  the Error of 87 is actually what is returned when you call Err.LastDllError.  when I call the code below from the Active X DLL this is what is returned:


I have pasted the declares and the function below, I have added in a few extra lines to test where the code is breaking and it seems that it is the first APi call (CryptAcquireContext).

Private Declare Function CryptAcquireContext Lib "advapi32.dll" Alias "CryptAcquireContextA" (ByRef phProv As Long, ByVal pszContainer As String, ByVal pszProvider As String, ByVal dwProvType As Long, ByVal dwFlags As Long) As Long

Private Declare Function CryptReleaseContext Lib "advapi32.dll" (ByVal hProv As Long, ByVal dwFlags As Long) As Long

Private Declare Function CryptCreateHash Lib "advapi32.dll" (ByVal hProv As Long, ByVal Algid As Long, ByVal hKey As Long, ByVal dwFlags As Long, ByRef phHash As Long) As Long

Private Declare Function CryptDestroyHash Lib "advapi32.dll" (ByVal hHash As Long) As Long

Private Declare Function CryptHashData Lib "advapi32.dll" (ByVal hHash As Long, pbData As Any, ByVal dwDataLen As Long, ByVal dwFlags As Long) As Long

Private Declare Function CryptGetHashParam Lib "advapi32.dll" (ByVal hHash As Long, ByVal dwParam As Long, pbData As Any, pdwDataLen As Long, ByVal dwFlags As Long) As Long

Private Const PROV_RSA_FULL = 1
Private Const ALG_CLASS_HASH = 32768
Private Const ALG_TYPE_ANY = 0
Private Const ALG_SID_MD2 = 1
Private Const ALG_SID_MD4 = 2
Private Const ALG_SID_MD5 = 3
Private Const ALG_SID_SHA1 = 4
Private Const HP_HASHVAL = 2
Private Const HP_HASHSIZE = 4

Public Function HashString(ByVal Str As String) As String
  Dim Algorithm: Algorithm = ALG_CLASS_HASH Or ALG_TYPE_ANY Or ALG_SID_MD5
  Dim hCtx As Long
  Dim hHash As Long
  Dim lRes As Long
  Dim lLen As Long
  Dim lIdx As Long
  Dim abData() As Byte

   HashString = HashString & "TEST1_" & lRes & "<BR>"
   lRes = CryptAcquireContext(hCtx, vbNullString, vbNullString, 1&, 0&)
   If lRes <> 0 Then
      HashString = HashString & "TEST2_" & lRes & "<BR>"
      lRes = CryptCreateHash(hCtx, Algorithm, 0, 0, hHash)
      If lRes <> 0 Then
         HashString = HashString & "TEST3_" & lRes & "<BR>"
         lRes = CryptHashData(hHash, ByVal Str, Len(Str), 0)
         If lRes <> 0 Then
            HashString = HashString & "TEST4_" & lRes & "<BR>"
            lRes = CryptGetHashParam(hHash, HP_HASHSIZE, lLen, 4, 0)
            If lRes <> 0 Then
                HashString = HashString & "TEST5_" & lRes & "<BR>"
                ReDim abData(0 To lLen - 1)
                lRes = CryptGetHashParam(hHash, HP_HASHVAL, abData(0), lLen, 0)
                If lRes <> 0 Then
                    HashString = HashString & "TEST6_" & lRes & "<BR>"
                    For lIdx = 0 To UBound(abData)
                        HashString = HashString & Right$("0" & Hex$(abData(lIdx)), 2)
                End If
            End If
         End If
         CryptDestroyHash hHash
      End If
   End If
   CryptReleaseContext hCtx, 0
   If lRes = 0 Then HashString = HashString & Err.LastDllError
End Function
Try doing your call to Err.LastDllError *before* CryptReleaseContext.  The error 87 at that point is probably because you don't have a valid context, therefore you're asking the API to release an invalid context.
DrackonAuthor Commented:

I see, I must have had a brain freeze not to see that one.  Thankyou so much!

The real error was #2 which I fixed by just adding the flag CRYPT_VERIFYCONTEXT as per the microsoft article:

Thanks Again!

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now