WinXP and VB 6 Format$

We are having trouble with the format$ command under Windows XP.

It has to do with rounding of numbers to 2 decimal places (eg Format$(ANumber, "0.00").  
If you pass in a number such as 1.23, it works fine.
If you pass in 1.265, the result is 1.27 - this is ok.
If you pass in 1.257, the result is 1.26 - this is ok.
If you pass in 1.253, the result is 1.25 - this is ok.

However:
If you pass in 1.235, the result is 1.23 - incorrect!
If you pass in 1.236, the result is 1.23 - incorrect!
If you pass in 1.239, the result is 1.23 - incorrect!

It appears to look at the second decimal place when evaluating the 3rd and subsequent places. If the 2nd number is < 5, then it ignores any possible rounding on the 3rd digit.

We have tried to "over ride" the format function by declaring our own, and if it meets any other criteria other than "0.00", we pass it out to VBA.Format$, otherwise we manipulate the code to round the number accordingly, but we think that this is a dangerous practice, and we are reluctant to release software with this in it.

Has anyone seen this, and if so, do they have an alternate solution?
finsterAsked:
Who is Participating?
 
TimCotteeConnect With a Mentor Head of Software ServicesCommented:
finster, the problem is due to the way "rounding" is implemented in VB. The following articles should help in understanding this:

http://support.microsoft.com/support/kb/articles/Q196/6/52.ASP

And

http://support.microsoft.com/support/kb/articles/Q194/9/83.ASP

0
 
RichWCommented:
Have you tried using the Round function?
Round(expression [,numdecimalplaces])

RichW
0
 
abelCommented:
I think there are several thinks at hand here that may cause (or help cause) this problem. Even though I could not find any specific information on this particular problem, I may have some thinks here that can be of help.

First of all, try a possible solution. If you are handy with API functions in VB, look up the documentation of GetNumberFormat. You can use it to do exactly what VB does. I think VB internally uses the same function, but if it doesn't, or if there's something else that goes wrong, you at least got that sorted out by using the API approach. And if it indeed does work correctly, you'll have a workaround as well.

Meanwhile I'll try to similate your situation, to see if I have the same problem on win2k (I'm sorry, I do not have winXP installed on any machine).

GetNumberFormat:
(I find the name misleading, it does not get the details of the format, you may give it. The function returns a formatted string, just as VB's Format$)
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/nls_8a0k.asp

HTH,
Cheers
Abel
0
 
abelCommented:
Hmm, I do not have the same problem on win2k, unfortunately, it appears to work correctly.

> I think there are several thinks at hand
I meant things, of course ;-)

Let's give that some thought. Basically, what happens here is that Format$, FormatNumber and GetNumberFormat all call into the same API. I do not know if Format$ and FormatNumber do any pre-formatting before passing the data to GetNumberFormat, but what I do know is that GetNumberFormat is part of the Internationalization API in Windows. You probably know the settings form in Control Panel, giving you all kind of options about numbers and all. I have never seen an option about rounding, but maybe something like that is added in XP? You may also want to try several other kinds of settings here, like increasing the number of digits in both the Currency and Number tabs or changing the locale country. To see of the settings are actually in effect in your VB app, simply change the Decimal Symbol on the Numbers tab to some strange symbol. You should see the change immediate in the Format$-output.

If all this doesn't help and the GetNumberFormat is to no avail either, I'm afraid you'll be stuck with using combinations of Fix, Round and the like.

Important note: Round() does not operate correctly on values like 1.255. It converts this value (incorrectly) to 1.25 instead of the correct 1.26.

HTH,
Cheers,
Abel
0
 
amebaCommented:
>If you pass in 1.236, the result is 1.23 - incorrect!
>If you pass in 1.239, the result is 1.23 - incorrect!

Are you sure?  There is a doubt and inconsistency if last digit is 5, but not for these:
Try in debug window:
? Format$(1.236, "0.00")
? Format$(1.239, "0.00")
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.