fulgeru99
asked on
Artimetic bug in .Net
It seems that .Net cannot make some simple numerical operations.
Try to write this in the debugger:
587.67 + 146.67
The result should be 734.34, but .Net calculates it to (586.67 + 146.67 = 733.33999999999992 Double)
Can somebody explain to me why?
I've found also other similar bugs but I think that showing one is enough.
Try to write this in the debugger:
587.67 + 146.67
The result should be 734.34, but .Net calculates it to (586.67 + 146.67 = 733.33999999999992 Double)
Can somebody explain to me why?
I've found also other similar bugs but I think that showing one is enough.
ASKER
Yes, if you write something like this is DISPLAYS correctly:
Sub Main()
Dim sum As Double = 587.67 + 146.67
Console.WriteLine(sum)
End Sub
But calling the WriteLine functions actualy calls Double.ToString() that formats the number and that seems to work. However, I am not using the ToString() function. I am calculating a sum in a foreach loop and then I am comparing it to another variable.
The problem is the way .Net internally stores that result.
Make the following checks also:
a) put a breakpoint on the line Console.WriteLine(sum) and add a watch for the variable sum
b) try this piece of code:
Sub Main()
Dim sum As Double = 587.67 + 146.67
Dim damnIt As Boolean = (734.34 = sum)
Console.WriteLine(damnIt)
End Sub
Sub Main()
Dim sum As Double = 587.67 + 146.67
Console.WriteLine(sum)
End Sub
But calling the WriteLine functions actualy calls Double.ToString() that formats the number and that seems to work. However, I am not using the ToString() function. I am calculating a sum in a foreach loop and then I am comparing it to another variable.
The problem is the way .Net internally stores that result.
Make the following checks also:
a) put a breakpoint on the line Console.WriteLine(sum) and add a watch for the variable sum
b) try this piece of code:
Sub Main()
Dim sum As Double = 587.67 + 146.67
Dim damnIt As Boolean = (734.34 = sum)
Console.WriteLine(damnIt)
End Sub
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
On this example, having the variable Decimal works.
However, I remember that I found a bug in Decimal also.
Anyway, try this: start debugging and add a new watch were you write: 587.674 + 146.67
No data type, just the calculation and see the result.
Moreover, type ?587.674 + 146.67 in the immediate output window.
Can anyone explain why?
I found that there are some special cases on some special sums where the calculation is wrong.
However, I remember that I found a bug in Decimal also.
Anyway, try this: start debugging and add a new watch were you write: 587.674 + 146.67
No data type, just the calculation and see the result.
Moreover, type ?587.674 + 146.67 in the immediate output window.
Can anyone explain why?
I found that there are some special cases on some special sums where the calculation is wrong.
This how this is showing in , but you will lose anything. this is how they storing as bits. in many times, if ur number is ending 1 4 or 8,you may get this lengthy result. But this is NOT true for many cases.
But you use round function, u will not lose anything.
Thad
But you use round function, u will not lose anything.
Thad
Where are you running this ? and what version of vs are you using ?