Solved

WinXP and VB 6 Format$

Posted on 2002-04-16
5
171 Views
Last Modified: 2010-05-02
We are having trouble with the format$ command under Windows XP.

It has to do with rounding of numbers to 2 decimal places (eg Format$(ANumber, "0.00").  
If you pass in a number such as 1.23, it works fine.
If you pass in 1.265, the result is 1.27 - this is ok.
If you pass in 1.257, the result is 1.26 - this is ok.
If you pass in 1.253, the result is 1.25 - this is ok.

However:
If you pass in 1.235, the result is 1.23 - incorrect!
If you pass in 1.236, the result is 1.23 - incorrect!
If you pass in 1.239, the result is 1.23 - incorrect!

It appears to look at the second decimal place when evaluating the 3rd and subsequent places. If the 2nd number is < 5, then it ignores any possible rounding on the 3rd digit.

We have tried to "over ride" the format function by declaring our own, and if it meets any other criteria other than "0.00", we pass it out to VBA.Format$, otherwise we manipulate the code to round the number accordingly, but we think that this is a dangerous practice, and we are reluctant to release software with this in it.

Has anyone seen this, and if so, do they have an alternate solution?
0
Comment
Question by:finster
5 Comments
 
LVL 4

Expert Comment

by:RichW
ID: 6946409
Have you tried using the Round function?
Round(expression [,numdecimalplaces])

RichW
0
 
LVL 39

Expert Comment

by:abel
ID: 6946415
I think there are several thinks at hand here that may cause (or help cause) this problem. Even though I could not find any specific information on this particular problem, I may have some thinks here that can be of help.

First of all, try a possible solution. If you are handy with API functions in VB, look up the documentation of GetNumberFormat. You can use it to do exactly what VB does. I think VB internally uses the same function, but if it doesn't, or if there's something else that goes wrong, you at least got that sorted out by using the API approach. And if it indeed does work correctly, you'll have a workaround as well.

Meanwhile I'll try to similate your situation, to see if I have the same problem on win2k (I'm sorry, I do not have winXP installed on any machine).

GetNumberFormat:
(I find the name misleading, it does not get the details of the format, you may give it. The function returns a formatted string, just as VB's Format$)
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/nls_8a0k.asp

HTH,
Cheers
Abel
0
 
LVL 39

Expert Comment

by:abel
ID: 6946454
Hmm, I do not have the same problem on win2k, unfortunately, it appears to work correctly.

> I think there are several thinks at hand
I meant things, of course ;-)

Let's give that some thought. Basically, what happens here is that Format$, FormatNumber and GetNumberFormat all call into the same API. I do not know if Format$ and FormatNumber do any pre-formatting before passing the data to GetNumberFormat, but what I do know is that GetNumberFormat is part of the Internationalization API in Windows. You probably know the settings form in Control Panel, giving you all kind of options about numbers and all. I have never seen an option about rounding, but maybe something like that is added in XP? You may also want to try several other kinds of settings here, like increasing the number of digits in both the Currency and Number tabs or changing the locale country. To see of the settings are actually in effect in your VB app, simply change the Decimal Symbol on the Numbers tab to some strange symbol. You should see the change immediate in the Format$-output.

If all this doesn't help and the GetNumberFormat is to no avail either, I'm afraid you'll be stuck with using combinations of Fix, Round and the like.

Important note: Round() does not operate correctly on values like 1.255. It converts this value (incorrectly) to 1.25 instead of the correct 1.26.

HTH,
Cheers,
Abel
0
 
LVL 43

Accepted Solution

by:
TimCottee earned 75 total points
ID: 6947168
finster, the problem is due to the way "rounding" is implemented in VB. The following articles should help in understanding this:

http://support.microsoft.com/support/kb/articles/Q196/6/52.ASP

And

http://support.microsoft.com/support/kb/articles/Q194/9/83.ASP

0
 
LVL 15

Expert Comment

by:ameba
ID: 6947299
>If you pass in 1.236, the result is 1.23 - incorrect!
>If you pass in 1.239, the result is 1.23 - incorrect!

Are you sure?  There is a doubt and inconsistency if last digit is 5, but not for these:
Try in debug window:
? Format$(1.236, "0.00")
? Format$(1.239, "0.00")
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
VB6 code to programmatically convert pdf to excel 21 70
using Access 8 59
passing parameter in sql procedure 9 56
VBA Shell can't Find Word document 11 76
Introduction In a recent article (http://www.experts-exchange.com/A_7811-A-Better-Concatenate-Function.html) for the Excel community, I showed an improved version of the Excel Concatenate() function.  While writing that article I realized that no o…
The debugging module of the VB 6 IDE can be accessed by way of the Debug menu item. That menu item can normally be found in the IDE's main menu line as shown in this picture.   There is also a companion Debug Toolbar that looks like the followin…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Get people started with the process of using Access VBA to control Excel using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Excel. Using automation, an Access application can laun…

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now