Link to home
Start Free TrialLog in
Avatar of peterchamberlin
peterchamberlin

asked on

Encryption within Registry (DBCS)

Hi all...

I have a problem with my application and DBCS operating system such as Japanese, Korean and Chinese Windows. Within SBCS Windows OS there is no problem within my program code, however, I have found that the encryption and decryption routines that I have coded for saving data reasonably securely do not work correctly under DBCS systems. It seems that the data gets written to the registry and is clipped before the actual end of information. I have expanded my encryption and decryption system to cope with the whole ASC() range of input values and I've also ensured that the encryption routine does not generated character 0 (which would act as a premature string terminator when writing the REG_SZ string).

Unfortunately I do not have access to a DBCS system myself and so debugging this issue is proving very difficult. The URL below is for a zip of my testing project which I'm using to try to overcome the problem.

http://www.geocities.com/peterchamberlin/dbcstest.zip

The encryption method I'm using is to take the input character number, get the value of this MOD 128, then scramble this number which will be in the range of 0-127. However, I'm leaving out the 0 value from the scrambling process so that values in the range of 1-127 can not get mapped on to 0. A string of random letters is use to obtain ASC numbers used to shift and modulate the input ASC number. The starting point of the constant containing the random letters is generated randomnly and appended to the end of the encrypted text. The text is encrypted by taking the last to first letter as input, to reverse the order of text as well. Finally the encrypted text is encapsulated in asterisks for identification of encryption.

I find that my encryption and decryption routines run fine within the VB IDE and also the VB EXE. However, when I introduce the writing of the resultant encrypted data to the registry, a read back from the registry and the decryption of this information then I find that the text is either garbled or invalid encrypted information.

Could someone please run and debug the test project for me on a DBCS enabled system to find and highlight the problem, or alternatively let me know the reason behind the registry issue. I cannot see anything in MSDN that would suggest such behaviour. VB itself (I'm using Vb6Sp5) deals throughout in Unicode and the Registry values should accept this and write to the registry accordingly. It may perhaps be my registry routines at fault, although I can't easily debug this.

Any help very much appreciated, as currently any encrypted data in my program running on DBCS systems gets corrupted between write and read-back, subsequently causing errors, crashes or validity check failures.
Avatar of Joebob
Joebob

Do you have access to Windows 2000 or NT 4?  If so then you have access to a DBCS.

When you call RegSetValueEx you must specify the cbSize of the string you are setting in the registry.  On a SBCS system, this size is simply the number of characters in the string plus 1 for the null terminator.  On a DBCS system, this size is calculated exactly the same way as a SBCS and then multiplied by 2.  If you don't handle this size difference, the data written to the registry will be cut in half on DBCS systems.
Avatar of peterchamberlin

ASKER

I don't have access to Win2k or NT4 here, although I do have a contact nearby who does.

I wasn't aware of the differing lengths when using RegSetValueEx. I'll look into that more, as that may be the reason why Windows is unable to write the data. However, the information that I've seen from snapshots doesn't look like half the length of the intended text, rather just the first few characters of encrypted information that seem to cut off, additionally missing the asterisks I've used to denote the encrypted information.

I'll send the revised test program to my contact and upload the revised code for others to check over.
I don't have access to Win2k or NT4 here, although I do have a contact nearby who does.

I wasn't aware of the differing lengths when using RegSetValueEx. I'll look into that more, as that may be the reason why Windows is unable to write the data. However, the information that I've seen from snapshots doesn't look like half the length of the intended text, rather just the first few characters of encrypted information that seem to cut off, additionally missing the asterisks I've used to denote the encrypted information.

I'll send the revised test program to my contact and upload the revised code for others to check over.

The users of my program who have Windows 2000 and NT4 have reported problems with the program, i.e. crashing on start-up. I was unable to resolve these at the time, although I think this problem may be at the root of them. If so then I should be able to debug this from the Win2k machine and just modify my registry write routine to resolve the matter.
Apologies for double post, accidentally hit Submit a little too early.

As quoted in MSDN the length field...

"Specifies the size, in bytes, of the information pointed to by the lpData parameter"

I'm currently using good old Len() to determine this, adding one for the vbNullChar at the end of the string. What I should be using is LenB().

I'll modify the program code accordingly and get my contact to check up on this. Annoying how I missed something so obvious, I guess I'm so used to thinking one byte per character. Haven't considered before whether API inputs were characters or bytes.
That is a common oversight, don't beat yourself up.  The APIs are sometimes tough to deal with and are ofter less than adequately documented.
Yep, I can see why it's a common oversight. I just haven't had to consider the DBCS side of things before. My previous software use file based storage for values and that worked without problems so it was a little strange to see things going wrong now.

I have adapted code to use LenB() and I'm still awaiting response from my contact. However, I'm a little uncertain that such a small change will fix the whole issue as from the snapshots of the registry test key I was using that he sent me it appears that between 1 and 5 characters are getting to the register under the Len() method, irrespective of the length of data that went through encryption and the registry write. Is it likely that the length being incorrect in DBCS could cause such a problem, or is the encryption itself perhaps likely to cause issues? I should know the answer shortly in any case. I have made sure that the encryption cannot generate new character 0's in the resultant cipher text, but perhaps there are other terminators within Japanese DBCS that cause the registry write to be cut short?

Another interesting thing is that if I use a variable name and assign it a string value, i.e. string into variant, then use Len and LenB on it, both of them return 8 (as I was using a 4 character string in the VB IDE). However, when I declared the variable as string with Dim, then set it I found that Len returned 4 as expected, but that LenB returned 8 as not expected. This is perhaps due to VB running Unicode throughout which is compulsory 2 bytes, but does this mean then that the registry routines have previously been working with half the amount of bytes storage space? Perhaps it's just been a case that all data I've written was converted by windows into SBCS when writing to the registry and so the length actually turned out correctly.

Hmm, lots to ponder...
Now that you mention it, VB has one DBCS feature that can be ugly.  VB, by default, converts all strings to ANSII (SBCS) prior to sending them to an API.  In your case, you need the API to receive the full DBCS otherwise data will be destroyed.  A work around I have had to make in a few cases is to change the string parameter in the API declaration to a long and pass a pointer to the actual string by using the undocumented VB function StrPtr.  This function will return a pointer to the first character of your string variable.

Note: With Windows NT, there are 2 sets of all the APIs, an ANSII version and a Unicode version.  An example is the below declare statement.  Notice that the Alias is "RegSetValueExA", this is the ANSII version, the Unicode version "RegSetValueExW".

Public Declare Function RegSetValueEx Lib "advapi32.dll" Alias "RegSetValueExA" (ByVal hKey As Long, ByVal lpValueName As String, ByVal Reserved As Long, ByVal dwType As Long, lpData As Any, ByVal cbData As Long) As Long
Yep, I thought that there might be something odd to do with the way that VB sent the data to the API. I'll try out the long pointer to the string method to see if that makes any difference.

I did read up on the note in MSDN about ANSI and Unicode calls, and the definitions I have in my source are all ANSI it seems, such as RegSetValueExA. Should I be using the Unicode version so that the character information gets written correctly all the time in both SBCS and DBSC? Will this effect the writing or reading of existing data that has gone to the registry in ANSI format? It would be rather a pain at present to switch the encoding used in my program's registry entries from ANSI to Unicode. Hopefully Windows will handle that on its own.

I'll let you know how the tests go.
All strings are stored in a fixed format in the registry.  This format is dependant on the OS.  One thing you will need to look at, the RegSetValueExW may not exist in the SBCS versions of Win9x.  If this is the case, you may need a separate version of your app for DBCS vs. SBCS.
If the RegSetValueEx API (ANSI version) uses the long pointer to the string then will the string contents still get converted to SBCS, or will the API deal with the data correctly? Converting the API to use StrPtr would work on both systems and I'd like if possible to be able to use the same executable on both SBCS and DBCS systems. There's probably a way to detect DBCS and so adjust the registry routines accordingly. From what you say, as long as the API receives the DBCS data correctly then the registry processing should work fine as the DBCS system will hold registry information in such a format. The question really is whether doing the Long StrPtr method will result in DBCS data getting to the registry, or whether the usage of RegSetValueExW must always be used. The case in hand though is a copy of Japanese Win98 SE. So this either means that RegSetValueExW exists for it, or the long ptr is sufficient.

Another thing is that my application seems to be writing data to the registry just fine for normal ASCII range, although I suspect this is due to the leading character not being part of the larger DBCS and so the data conversion to ANSI doesn't affect the information. The encryption results in characters shuffled around in blocks of 128 across the whole character number range though.
VB automatically converts it's internal Unicode strings into ANSI strings when passed as strings to API functions regardless of whether the API is a Unicode or ANSI function.  This is a limitation of VB written to ensure it is compatible with the US versions of Win9x.  You can overide this behavior by passing the parameters as long and using the StrPtr function.  The catch is that you can't do this with the ANSI version of the functions because they are expecting a SBCS.  You must use the Unicode version of the functions.  This requirement is why I mentioned you may have to have different versions of your app since it is likely that the SBCS versions of Win9x do not include the unicode versions of the function.
Ah, I see now. The StrPtr Long is only effectual when using Unicode function versions, as both provide DBCS whereas ANSI will only take SBCS. This still leaves the problem though that the Japanese Win98 machine is having problems, as I will not be able to use Unicode functions on that (presumably) and so that leaves just the Len/LenB thing to be the sole cause of problems. The annoying thing is that this bug also occurs on NT4 and certain Win2000 installations, which I'd like to fix ready for distribution on networks.
This also occurs in US versions NT4 and Win2k?
Apparently so, they get exactly the same symptoms of the crash that occurs on Japanese Win98, and this is the result of the registration encryption/decryption not working properly. It's got to be down to the registry writing routines as I'm certain that the output from the encryption and decryption routines works fine. The US systems that the crash occurs on all support DBCS.
With a US DBCS System (i.e. WinNT and 2k), the length thing is not a problem because VB can successfully convert the string into an ANSI string (SBCS) prior to sending to the API.  This is because the US Alphabet does not require the second byte in the DBCS.  Because of what I just said, I can't understand why it would fail in the US.  Have you inserted any error handling and reporting to confirm exactly where the failure is happening.  If the problem is actually creating an error that can be trapped using an on error goto statement, try adding line numbers to your code.  If your code has line numbers, you can use the undocumented variable ERL in your error handling routine that will give the line that caused the error.
That's interesting, I haven't heard of ERL before and that could be useful in sorting out problems in the future. Unfortunately the problem only shows up as a run-time crash and that's because of a type error during a subclassed routine when the decrypted information coming back from the registry is attempted to be converted into a date. I cannot simply error trap and continue as this data is critical to the shareware registration locking system.

I will request a copy of the registry entries on the US machines that have the problem and see if it matches up to what I've experienced with the Japanese installation.

The last test using LenB came back with the reg entry showing some Japanese characters however the data didn't begin with the correct asterisk prefix which would suggest clipping before the end of the string as well as at the end. I've written a new test program that I've sent off to be tried out that has a unicode registry write routine using the RegSetValueExW API call, using the long pointer to string and StrPtr and also using StrConv(Data, vbWide) for all other API parameters. I'm awaiting results on that one currently.
Well, I've tried a few more things with Jap Win98 and it turns out that the Unicode functions don't work correctly (i.e. the value written to registry just doesn't appear, even though the function passes correctly).

Do you perhaps have some sample code of a registry write and read on a DBCS system that I could use for comparison? I've run out of options now for solving this problem.

My Japanese contact reported that the last trial edition of my test program worked for encrypted write/read, this being when the test data was generated by using ASCII 32-126 from the Japanese character set. This would suggest that the encrypt/write/read/decrypt system appears to be working, but I'm still skeptical. Awaiting a reply after they have tested the program on the next day, to see whether the encryption system really has worked correctly.

It's annoying to be handling an issue that is not definite.
It makes sense that it would work in the ASCII 32-126 range.  ASCII 32-126 is still a single byte character, you don't get into double byte characters until 256.  VB esentially truncates the second byte of a DBCS when it converts from DBCS to SBCS.  In the case of ASCII 32-126, the second byte is binary 0 so truncated it doesn't change the value.  In the case of ASCII 256, the second byte is binary 1 and when you trancate 256 to only 1 byte, you get ASCII 255 (data corruption).  If you can control your encrytion algorithm so that it never will use a character outside the 0-255 range, then DBCS shouldn't matter.
The encryption system I'm using has been set to scramble things in blocks of 128, i.e. 0-127 (excluding 0), 128-255 etc. So that's perhaps why the scrambled text isn't causing problems anymore. The thing is that the user may enter DBCS data for their passwords, and when these get encrypted they will be scrambled within the range but the written data will be lost or corrupted as usual. I can't easily implement a unicode routine as I have yet to get such a routine operational. More Win2k testing time I think.

If you still have a unicode registry write/read routine then I'd appreciate quickly reading over the source code. I would think it's the same thing, just substituting for the W suffix and using StrConv("data", vbWide) for any other API parameters.
I don't have any Unicode code writting to the registry, but Karl E. Peterson has an example of passing a string pointer to an Unicode api.  To see his example go to http://www.mvps.org/vb/samples.htm and download the UNCName project.  This project uses the netapis, which under NT, will only except unicode strings.
I don't have any Unicode code writting to the registry, but Karl E. Peterson has an example of passing a string pointer to an Unicode api.  To see his example go to http://www.mvps.org/vb/samples.htm and download the UNCName project.  This project uses the netapis, which under NT, will only except unicode strings.

You will need to search the code for a call to the function StrPtr.  This function returns a pointer to the string that can then be passed as a long to an API.
ASKER CERTIFIED SOLUTION
Avatar of Joebob
Joebob

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Ah, I've been using lpData as Long for the StrPtr() parameter, and hadn't been using StrPtr for the value name as it's only been the value data that hasn't been writing correctly.

I'll try out the modified Unicode on the Jap Win98 machine however the last attempt just didn't do anything with the Unicode API call, although that might be the default behaviour if VB passes all strings to it as SBCS.
If the ValueName has been working fine then you shouldn't need to use the StrPtr function for it.  The lpData parameter should be like this:

lpData = StrPtr(sData)
RegSetValueEx_UNICODE(hkey,lpValueName,Reserved,dwType,byval lpData,cbData)

The byval is necessary otherwise vb will send a pointer to lpData which is a pointer to your string ( a pointer to a pointer), but the API doesn't expect to have to step through 2 pointers.
*** IGNORE MY LAST POST ***
*** MY BRAIN ISN'T WORKING RIGHT ***

You need to use StrPtr on the ValueName parameter because the API is expecting a Unicode string, but VB is going to automatically convert the string to ANSI prior to calling the API.  The data also must be passed using the StrPtr function, again so VB doesn't automatically convert it to ANSI.

lpData = StrPtr(sData)
RegSetValueEx_UNICODE(hkey,StrPtr(lpValueName),Reserved,dwType,ByVal lpData,cbData)

The byval is necessary otherwise vb will send a pointer to lpData which is a pointer to your string
( a pointer to a pointer), but the API doesn't expect to have to step through 2 pointers.
So shouldn't the StrPtr(lpValueName) also be ByVal, and I presume in the API function definition the key name would be set up to use lpValueName as Long or as Any.

I'll update my code to use this method and see whether any luck is to be had. If it does work, and works on Jap Win98 (which I suspect must be the case for a DBCS OS) then all I then need is a way to determine whether the system my app runs on is SBCS or DBCS.
If you look at my RegSetValueEx_UNICODE definition, I have the lpValueName param declared byval as long already.  You need the byval on the lpData because it is declared byref as any in the api declaration.
I now have...

Private Declare Function RegSetValueEx2 Lib "advapi32.dll" _
      Alias "RegSetValueExW" _
      (ByVal hKey As Long, ByVal lpValueName As Long, _
      ByVal Reserved As Long, ByVal dwType As Long, _
      ByVal lpData As Long, ByVal cbData As Long) As Long

and...

lResult = RegSetValueEx2(lHandle, ByVal StrPtr(sValueName), 0, REG_SZ, ByVal StrPtr(sValue), lLenData)

Will let you know how this tests under Win98 English and Japanese. I still suspect that the Win98 Japanese OS supports Unicode API, as it needs such for all the DBCS work.
Avatar of DanRollins
Hi peterchamberlin,
It appears that you have forgotten this question. I will ask Community Support to close it unless you finalize it within 7 days. I will ask a Community Support Moderator to:

    Accept Joebob's comment(s) as an answer.

peterchamberlin, if you think your question was not answered at all or if you need help, just post a new comment here; Community Support will help you.  DO NOT accept this comment as an answer.

EXPERTS: If you disagree with that recommendation, please post an explanatory comment.
==========
DanRollins -- EE database cleanup volunteer
At the time there wasn't any suitable response as to how the problem could be resolved. I did manage it on my own in the end by utilising the following code...

' Calculate Length in Bytes
If sValue = "" Then
   ' Special Case Handling
   lLenData = 2
   sValue = vbNullChar
Else
   lLenData = LenB(sValue & vbNullChar)
End If

lResult = RegSetValueEx(lHandle, sValueName, 0, REG_SZ, ByVal sValue, lLenData)

The issue was all down to counting the number of bytes correctly as can be seen above. I've also had to add more code to deal with the special case of an empty string as doing the above procedure on WinXP without it caused the whole application to crash resulting in an OS level (unhelpful) error message. It seems to be an over cautious thing on behalf of Micro$oft to be more careful as to what gets posted to the registry. Possibly a countermeasure to some kind of attack.

As Joe Bob was the most helpful poster then I'm okay with the points going to him.
Hi peterchamberlin,
Thanks for returning to this year old question.  To award points to Joebob, please click the [Accept Comment as Answer] button next to one of his comments.  Then select the Excellent radio button and then click [Submit].  Once you get the hang of it, I'm sure you will find it quite easy to finailze questions in the future.  Thanks!
-- Dan
I am familiar with the answering process, just forgot to award the points after typing the last comment. Normally I do follow through with all my posts, must just have been quite busy at that particular time.