asked on

CryptAPI Not Working in ActiveX DLL

Hi,

I am trying to write an ActiveX DLL that makes a call to the CryptAcquireContext API. Everything works perfectly when I cut and paste the code directly into an executable and run it on the same machine.

The ActiveX DLL is being called from an asp script on a web server and it is failing with code 87

The API call is:
lRet = CryptAcquireContext(hCtx, "", "", 1, 0)

Thanks

PS: Please don't anybody give me a link to a class or dll that does encryption for me. I specifically want to hash a string from within this DLL and not using massive code overhead that I don't need. I'm only interested in why this isn't working and how to get around it (or in an easy to implement hash algorithm if anyone has one)

aParser

Error 87 is "The parameter is incorrect."

In looking over your parameters, I would have expected you to use vbNullString instead of "". The 1 (PROV_RSA_FULL) looks fine, although I might explicitly qualify it with a typechar (i.e., 1&) just to make sure it was full size. Ditto for the Flag parameter (0). I can only assume that the hCtx is Dim'ed properly.

Other than that, you might post the Declare statement that you're using. It's probably fine, since you say that it's working in an EXE. Along those lines, since the Container and Provider parameters are pointers to strings, if I were in your shoes and it still wasn't working, I might alter the Declare to change them from String to Long and pass zeros (since it appears you're trying to pass NULL for those parameters).

Honestly, given the error message and the line of code, I'd really suspect the empty string as being the culprit. If you can, give vbNullString a try and, if that doesn't work, post back here with your Declare.

And just to make sure: the error 87 is the value of lRet which is in the DLL, not in the ASP, right?

Drackon

ASKER

I have tried the vbNullString and the qualifying as suggested and altering the Declare. the Error of 87 is actually what is returned when you call Err.LastDllError. when I call the code below from the Active X DLL this is what is returned:

TEST1_0 87

I have pasted the declares and the function below, I have added in a few extra lines to test where the code is breaking and it seems that it is the first APi call (CryptAcquireContext).

Private Declare Function CryptAcquireContext Lib "advapi32.dll" Alias "CryptAcquireContextA" (ByRef phProv As Long, ByVal pszContainer As String, ByVal pszProvider As String, ByVal dwProvType As Long, ByVal dwFlags As Long) As Long

Private Declare Function CryptReleaseContext Lib "advapi32.dll" (ByVal hProv As Long, ByVal dwFlags As Long) As Long

Private Declare Function CryptCreateHash Lib "advapi32.dll" (ByVal hProv As Long, ByVal Algid As Long, ByVal hKey As Long, ByVal dwFlags As Long, ByRef phHash As Long) As Long

Private Declare Function CryptDestroyHash Lib "advapi32.dll" (ByVal hHash As Long) As Long

Private Declare Function CryptHashData Lib "advapi32.dll" (ByVal hHash As Long, pbData As Any, ByVal dwDataLen As Long, ByVal dwFlags As Long) As Long

Private Declare Function CryptGetHashParam Lib "advapi32.dll" (ByVal hHash As Long, ByVal dwParam As Long, pbData As Any, pdwDataLen As Long, ByVal dwFlags As Long) As Long

Private Const PROV_RSA_FULL = 1
Private Const ALG_CLASS_HASH = 32768
Private Const ALG_TYPE_ANY = 0
Private Const ALG_SID_MD2 = 1
Private Const ALG_SID_MD4 = 2
Private Const ALG_SID_MD5 = 3
Private Const ALG_SID_SHA1 = 4
Private Const HP_HASHVAL = 2
Private Const HP_HASHSIZE = 4

Public Function HashString(ByVal Str As String) As String
Dim Algorithm: Algorithm = ALG_CLASS_HASH Or ALG_TYPE_ANY Or ALG_SID_MD5
Dim hCtx As Long
Dim hHash As Long
Dim lRes As Long
Dim lLen As Long
Dim lIdx As Long
Dim abData() As Byte

HashString = HashString & "TEST1_" & lRes & " "
lRes = CryptAcquireContext(hCtx, vbNullString, vbNullString, 1&, 0&)
If lRes <> 0 Then
HashString = HashString & "TEST2_" & lRes & " "
lRes = CryptCreateHash(hCtx, Algorithm, 0, 0, hHash)
If lRes <> 0 Then
HashString = HashString & "TEST3_" & lRes & " "
lRes = CryptHashData(hHash, ByVal Str, Len(Str), 0)
If lRes <> 0 Then
HashString = HashString & "TEST4_" & lRes & " "
lRes = CryptGetHashParam(hHash, HP_HASHSIZE, lLen, 4, 0)
If lRes <> 0 Then
HashString = HashString & "TEST5_" & lRes & " "
ReDim abData(0 To lLen - 1)
lRes = CryptGetHashParam(hHash, HP_HASHVAL, abData(0), lLen, 0)
If lRes <> 0 Then
HashString = HashString & "TEST6_" & lRes & " "
For lIdx = 0 To UBound(abData)
HashString = HashString & Right$("0" & Hex$(abData(lIdx)), 2)
Next
End If
End If
End If
CryptDestroyHash hHash
End If
End If
CryptReleaseContext hCtx, 0
If lRes = 0 Then HashString = HashString & Err.LastDllError
End Function

ASKER CERTIFIED SOLUTION

aParser

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Drackon

ASKER

Aaaaahh...

I see, I must have had a brain freeze not to see that one. Thankyou so much!

The real error was #2 which I fixed by just adding the flag CRYPT_VERIFYCONTEXT as per the microsoft article:
http://support.microsoft.com/default.aspx?scid=238187

Thanks Again!