Solved

Unmanaged C DLL calling from VB.net and having it pass wrong information

Posted on 2004-10-06
2
264 Views
Last Modified: 2013-11-20
I have a c++ dll I created that encrypts a string and converts the string to HEX and vice versa.

Here is the following functions from my dll file (c++)

void * __stdcall CreateDll3()
void __stdcall DestroyDll3(void * objptr)
int __stdcall dEncrypt(void * objptr, char* sData, char* sPassword)
int __stdcall dDecrypt(void * objptr,char *sData, char *sPassword)
BSTR __stdcall dGetBuffer(void * objptr)
void __stdcall ExecuteProgram(BSTR cmdl,BSTR startDir)
BSTR __stdcall CharStr2HexStr(char* pucCharStr, int iSize)
BSTR __stdcall HexStr2CharStr(char* pszHexStr, int iSize)

Now from my vb.net program I am doing the following

    Private Declare Function CreateDll3 Lib "Encryption.dll" () As Long
    Private Declare Sub DestroyDll3 Lib "Encryption.dll" (ByVal objptr As Long)
    Private Declare Function dGetBuffer Lib "Encryption.dll" (ByVal objptr As Long) As String
    Private Declare Function dDecrypt Lib "Encryption.dll" (ByVal objptr As Long, ByVal qData As String, ByVal password1 As String) As Integer
    Private Declare Function dEncrypt Lib "Encryption.dll" (ByVal objptr As Long, ByVal qData As String, ByVal password1 As String) As Integer
    Private Declare Sub ExecuteProgram Lib "Encryption.dll" (ByVal cmdline As String, ByVal startDir As String)
    Private Declare Function CharStr2HexStr Lib "Encryption.dll" (ByVal str As String, ByVal size As Integer) As String
    Private Declare Function HexStr2CharStr Lib "Encryption.dll" (ByVal hexcode As String, ByVal size As Integer) As String

    Private Sub mnuCreateSession_Click_1(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles mnuCreateSession.Click
        Dim path As String
        Dim tagetApp As String
        Dim encrypt As String
        Dim objptr As Long
        Dim enc As String
        Dim peram As String
        Dim hexcode As String

        objptr = CreateDll3()
        'txtPassword.Text = vPass
        enc = dEncrypt(objptr, vPass, "123456789")
        encrypt = dGetBuffer(objptr)
        hexcode = CharStr2HexStr(encrypt, 16)
        DestroyDll3(objptr)

        MsgBox(encrypt)
        peram = vUserName + " " + hexcode + " " + vSystem + " " + vCli

        path = Application.StartupPath + "\AppLoader.exe"

        Dim psi As New System.Diagnostics.ProcessStartInfo(path)
        psi.WorkingDirectory = Application.StartupPath
        MsgBox(peram)
        psi.Arguments = "vbquicksd" + " " + peram
        psi.RedirectStandardOutput = False
        psi.WindowStyle = ProcessWindowStyle.Normal
        psi.UseShellExecute = False
        Dim ShareGuard As System.Diagnostics.Process
        Try
            ShareGuard = System.Diagnostics.Process.Start(psi)
        Catch ex As Exception
            MsgBox(ex.Message)
        End Try

    End Sub

My issue is when you call the dEncrypt function you pass the object of the class Crypto, the users password and your encryption shared key. Everything seems to pass correctly except in the dll the function is recieving a werd string of characters as the users password and the users password is ending up as the encryption password. I also wrote this same thing in VB 6 and everything works great, so I know it has something to do with .NET.
0
Comment
Question by:rader19
2 Comments
 
LVL 48

Accepted Solution

by:
AlexFM earned 500 total points
ID: 12246111
Try this:

Private Declare Function dDecrypt Lib "Encryption.dll"
    (ByVal objptr As Long,
    <MarshalAs(UnmanagedType.LPStr)>
    ByVal qData As String,
    <MarshalAs(UnmanagedType.LPStr)>
    ByVal password1 As String) As Integer
0

Featured Post

Secure Your Active Directory - April 20, 2017

Active Directory plays a critical role in your company’s IT infrastructure and keeping it secure in today’s hacker-infested world is a must.
Microsoft published 300+ pages of guidance, but who has the time, money, and resources to implement? Register now to find an easier way.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Strange date stored 22 176
Named range not carried over 10 66
x-bar in Google Sheets 2 97
Authenticate using sesu from script 7 117
In this article, I'll describe -- and show pictures of -- some of the significant additions that have been made available to programmers in the MFC Feature Pack for Visual C++ 2008.  These same feature are in the MFC libraries that come with Visual …
If you use Adobe Reader X it is possible you can't open OLE PDF documents in the standard. The reason is the 'save box mode' in adobe reader X. Many people think the protected Mode of adobe reader x is only to stop the write access. But this fe…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
A short tutorial showing how to set up an email signature in Outlook on the Web (previously known as OWA). For free email signatures designs, visit https://www.mail-signatures.com/articles/signature-templates/?sts=6651 If you want to manage em…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question