Solved

WinXP and VB 6 Format$

Posted on 2002-04-16
5
170 Views
Last Modified: 2010-05-02
We are having trouble with the format$ command under Windows XP.

It has to do with rounding of numbers to 2 decimal places (eg Format$(ANumber, "0.00").  
If you pass in a number such as 1.23, it works fine.
If you pass in 1.265, the result is 1.27 - this is ok.
If you pass in 1.257, the result is 1.26 - this is ok.
If you pass in 1.253, the result is 1.25 - this is ok.

However:
If you pass in 1.235, the result is 1.23 - incorrect!
If you pass in 1.236, the result is 1.23 - incorrect!
If you pass in 1.239, the result is 1.23 - incorrect!

It appears to look at the second decimal place when evaluating the 3rd and subsequent places. If the 2nd number is < 5, then it ignores any possible rounding on the 3rd digit.

We have tried to "over ride" the format function by declaring our own, and if it meets any other criteria other than "0.00", we pass it out to VBA.Format$, otherwise we manipulate the code to round the number accordingly, but we think that this is a dangerous practice, and we are reluctant to release software with this in it.

Has anyone seen this, and if so, do they have an alternate solution?
0
Comment
Question by:finster
5 Comments
 
LVL 4

Expert Comment

by:RichW
Comment Utility
Have you tried using the Round function?
Round(expression [,numdecimalplaces])

RichW
0
 
LVL 39

Expert Comment

by:abel
Comment Utility
I think there are several thinks at hand here that may cause (or help cause) this problem. Even though I could not find any specific information on this particular problem, I may have some thinks here that can be of help.

First of all, try a possible solution. If you are handy with API functions in VB, look up the documentation of GetNumberFormat. You can use it to do exactly what VB does. I think VB internally uses the same function, but if it doesn't, or if there's something else that goes wrong, you at least got that sorted out by using the API approach. And if it indeed does work correctly, you'll have a workaround as well.

Meanwhile I'll try to similate your situation, to see if I have the same problem on win2k (I'm sorry, I do not have winXP installed on any machine).

GetNumberFormat:
(I find the name misleading, it does not get the details of the format, you may give it. The function returns a formatted string, just as VB's Format$)
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/nls_8a0k.asp

HTH,
Cheers
Abel
0
 
LVL 39

Expert Comment

by:abel
Comment Utility
Hmm, I do not have the same problem on win2k, unfortunately, it appears to work correctly.

> I think there are several thinks at hand
I meant things, of course ;-)

Let's give that some thought. Basically, what happens here is that Format$, FormatNumber and GetNumberFormat all call into the same API. I do not know if Format$ and FormatNumber do any pre-formatting before passing the data to GetNumberFormat, but what I do know is that GetNumberFormat is part of the Internationalization API in Windows. You probably know the settings form in Control Panel, giving you all kind of options about numbers and all. I have never seen an option about rounding, but maybe something like that is added in XP? You may also want to try several other kinds of settings here, like increasing the number of digits in both the Currency and Number tabs or changing the locale country. To see of the settings are actually in effect in your VB app, simply change the Decimal Symbol on the Numbers tab to some strange symbol. You should see the change immediate in the Format$-output.

If all this doesn't help and the GetNumberFormat is to no avail either, I'm afraid you'll be stuck with using combinations of Fix, Round and the like.

Important note: Round() does not operate correctly on values like 1.255. It converts this value (incorrectly) to 1.25 instead of the correct 1.26.

HTH,
Cheers,
Abel
0
 
LVL 43

Accepted Solution

by:
TimCottee earned 75 total points
Comment Utility
finster, the problem is due to the way "rounding" is implemented in VB. The following articles should help in understanding this:

http://support.microsoft.com/support/kb/articles/Q196/6/52.ASP

And

http://support.microsoft.com/support/kb/articles/Q194/9/83.ASP

0
 
LVL 15

Expert Comment

by:ameba
Comment Utility
>If you pass in 1.236, the result is 1.23 - incorrect!
>If you pass in 1.239, the result is 1.23 - incorrect!

Are you sure?  There is a doubt and inconsistency if last digit is 5, but not for these:
Try in debug window:
? Format$(1.236, "0.00")
? Format$(1.239, "0.00")
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Introduction I needed to skip over some file processing within a For...Next loop in some old production code and wished that VB (classic) had a statement that would drop down to the end of the current iteration, bypassing the statements that were c…
When trying to find the cause of a problem in VBA or VB6 it's often valuable to know what procedures were executed prior to the error. You can use the Call Stack for that but it is often inadequate because it may show procedures you aren't intereste…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now