[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 274
  • Last Modified:

Unmanaged C DLL calling from VB.net and having it pass wrong information

I have a c++ dll I created that encrypts a string and converts the string to HEX and vice versa.

Here is the following functions from my dll file (c++)

void * __stdcall CreateDll3()
void __stdcall DestroyDll3(void * objptr)
int __stdcall dEncrypt(void * objptr, char* sData, char* sPassword)
int __stdcall dDecrypt(void * objptr,char *sData, char *sPassword)
BSTR __stdcall dGetBuffer(void * objptr)
void __stdcall ExecuteProgram(BSTR cmdl,BSTR startDir)
BSTR __stdcall CharStr2HexStr(char* pucCharStr, int iSize)
BSTR __stdcall HexStr2CharStr(char* pszHexStr, int iSize)

Now from my vb.net program I am doing the following

    Private Declare Function CreateDll3 Lib "Encryption.dll" () As Long
    Private Declare Sub DestroyDll3 Lib "Encryption.dll" (ByVal objptr As Long)
    Private Declare Function dGetBuffer Lib "Encryption.dll" (ByVal objptr As Long) As String
    Private Declare Function dDecrypt Lib "Encryption.dll" (ByVal objptr As Long, ByVal qData As String, ByVal password1 As String) As Integer
    Private Declare Function dEncrypt Lib "Encryption.dll" (ByVal objptr As Long, ByVal qData As String, ByVal password1 As String) As Integer
    Private Declare Sub ExecuteProgram Lib "Encryption.dll" (ByVal cmdline As String, ByVal startDir As String)
    Private Declare Function CharStr2HexStr Lib "Encryption.dll" (ByVal str As String, ByVal size As Integer) As String
    Private Declare Function HexStr2CharStr Lib "Encryption.dll" (ByVal hexcode As String, ByVal size As Integer) As String

    Private Sub mnuCreateSession_Click_1(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles mnuCreateSession.Click
        Dim path As String
        Dim tagetApp As String
        Dim encrypt As String
        Dim objptr As Long
        Dim enc As String
        Dim peram As String
        Dim hexcode As String

        objptr = CreateDll3()
        'txtPassword.Text = vPass
        enc = dEncrypt(objptr, vPass, "123456789")
        encrypt = dGetBuffer(objptr)
        hexcode = CharStr2HexStr(encrypt, 16)
        DestroyDll3(objptr)

        MsgBox(encrypt)
        peram = vUserName + " " + hexcode + " " + vSystem + " " + vCli

        path = Application.StartupPath + "\AppLoader.exe"

        Dim psi As New System.Diagnostics.ProcessStartInfo(path)
        psi.WorkingDirectory = Application.StartupPath
        MsgBox(peram)
        psi.Arguments = "vbquicksd" + " " + peram
        psi.RedirectStandardOutput = False
        psi.WindowStyle = ProcessWindowStyle.Normal
        psi.UseShellExecute = False
        Dim ShareGuard As System.Diagnostics.Process
        Try
            ShareGuard = System.Diagnostics.Process.Start(psi)
        Catch ex As Exception
            MsgBox(ex.Message)
        End Try

    End Sub

My issue is when you call the dEncrypt function you pass the object of the class Crypto, the users password and your encryption shared key. Everything seems to pass correctly except in the dll the function is recieving a werd string of characters as the users password and the users password is ending up as the encryption password. I also wrote this same thing in VB 6 and everything works great, so I know it has something to do with .NET.
0
rader19
Asked:
rader19
1 Solution
 
AlexFMCommented:
Try this:

Private Declare Function dDecrypt Lib "Encryption.dll"
    (ByVal objptr As Long,
    <MarshalAs(UnmanagedType.LPStr)>
    ByVal qData As String,
    <MarshalAs(UnmanagedType.LPStr)>
    ByVal password1 As String) As Integer
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now