This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

It seems that .Net cannot make some simple numerical operations.

Try to write this in the debugger:

587.67 + 146.67

The result should be 734.34, but .Net calculates it to (586.67 + 146.67 = 733.33999999999992 Double)

Can somebody explain to me why?

I've found also other similar bugs but I think that showing one is enough.

Try to write this in the debugger:

587.67 + 146.67

The result should be 734.34, but .Net calculates it to (586.67 + 146.67 = 733.33999999999992 Double)

Can somebody explain to me why?

I've found also other similar bugs but I think that showing one is enough.

Where are you running this ? and what version of vs are you using ?

Sub Main()

Dim sum As Double = 587.67 + 146.67

Console.WriteLine(sum)

End Sub

But calling the WriteLine functions actualy calls Double.ToString() that formats the number and that seems to work. However, I am not using the ToString() function. I am calculating a sum in a foreach loop and then I am comparing it to another variable.

The problem is the way .Net internally stores that result.

Make the following checks also:

a) put a breakpoint on the line Console.WriteLine(sum) and add a watch for the variable sum

b) try this piece of code:

Sub Main()

Dim sum As Double = 587.67 + 146.67

Dim damnIt As Boolean = (734.34 = sum)

Console.WriteLine(damnIt)

End Sub

Take a look at some of the references/toys here: http://babbage.cs.qc.edu/IEEE-754/

This is definitely not unique to .NET; it's common to how all computers process floating-point numbers.

As Sancler suggests, the workaround when decimal precison matters the Decimal format is more appropriate. It makes no attempt to convert your decimal floating-point numbers to binary floating-point numbers, but rather stores each digit separately. There is, thus, no chance of rounding errors in conversion.

However, I remember that I found a bug in Decimal also.

Anyway, try this: start debugging and add a new watch were you write: 587.674 + 146.67

No data type, just the calculation and see the result.

Moreover, type ?587.674 + 146.67 in the immediate output window.

Can anyone explain why?

I found that there are some special cases on some special sums where the calculation is wrong.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.

Dim damnIt As Boolean = (734.34 = sum.ToString)

- would be faster), it sounds like you want to use the Decimal datatype rather than the Double.

From the documentation

>>

The Decimal data type provides the greatest number of significant digits for a number. It supports up to 29 significant digits and can represent values in excess of 7.9228 x 10^28. It is particularly suitable for calculations, such as financial, that require a large number of digits but cannot tolerate rounding errors.

The default value of Decimal is 0.

Programming Tips

Precision. Decimal is not a floating-point data type. The Decimal structure holds a binary integer value, together with a sign bit and an integer scaling factor that specifies what portion of the value is a decimal fraction. Because of this, Decimal numbers have a more precise representation in memory than floating-point types (Single and Double).

Performance. The Decimal data type is the slowest of all the numeric types. You should weigh the importance of precision against performance before choosing a data type.

<<

Roger