Jason-
asked on
Incorrect Math In C#
VS 2008 Standard - C# - Windows Forms Application
Double packagecharge = 7.56;
Double cnt = 3;
Double percost = packagecharge / cnt;
Double diff = packagecharge - (percost * cnt);
The result of diff should evalute to 0 but instead it evalutes to -0.00000000000000088817841
When i break in my code and move my mouse over the (percost * cnt) expression it shows me the result of that as being 7.56000000000005.. so thats why diff is incorrect.
Is there some compiler setting i need to enable?
I am lost...
Double packagecharge = 7.56;
Double cnt = 3;
Double percost = packagecharge / cnt;
Double diff = packagecharge - (percost * cnt);
The result of diff should evalute to 0 but instead it evalutes to -0.00000000000000088817841
When i break in my code and move my mouse over the (percost * cnt) expression it shows me the result of that as being 7.56000000000005.. so thats why diff is incorrect.
Is there some compiler setting i need to enable?
I am lost...
peetm
Round it - just *you* do with PI!
Note that your perfectly precise decimal fraction 0.56 = 56/100 = 14/25
can only be approximated by a binary fraction.
-0.00000000000000088817841
can only be approximated by a binary fraction.
-0.00000000000000088817841
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
A double is stored in 8 bytes, which makes 64bits.
Of these, one is needed for the sign, 11 are used for an exponent and the remaining 52 bits for the mantissa (cf. http://en.wikipedia.org/wiki/IEEE_754-1985), This implies that a relative error of 2^(-52) is unavoidable.
Without respecting some specific details, we can conclude that the biggiest possible number is about 2^2048 (approx. 10^616) times bigger than the smallest possible number because 2048 different exponents are possible, and since 2^{-52} approx. 10^{-15}, at most 15-16 decimal digits of precision can be achieved.
Try this:
Double a = 1e20;
Double b = 123456;
Double diff = (a+b)-a;
Try the same with a=1e15 and a=1e30. You will note that due to rounding error, the result is not b in all cases.
Also, since the rounding occurs in the binary system, the number 123456 will not degrade to 123460, then 123500, then 123000, then 120000, then 100000 (i.e. multiples of bigger and bigger powers of ten), but rather to some multiples of bigger and bigger powers of 2.
Of these, one is needed for the sign, 11 are used for an exponent and the remaining 52 bits for the mantissa (cf. http://en.wikipedia.org/wiki/IEEE_754-1985), This implies that a relative error of 2^(-52) is unavoidable.
Without respecting some specific details, we can conclude that the biggiest possible number is about 2^2048 (approx. 10^616) times bigger than the smallest possible number because 2048 different exponents are possible, and since 2^{-52} approx. 10^{-15}, at most 15-16 decimal digits of precision can be achieved.
Try this:
Double a = 1e20;
Double b = 123456;
Double diff = (a+b)-a;
Try the same with a=1e15 and a=1e30. You will note that due to rounding error, the result is not b in all cases.
Also, since the rounding occurs in the binary system, the number 123456 will not degrade to 123460, then 123500, then 123000, then 120000, then 100000 (i.e. multiples of bigger and bigger powers of ten), but rather to some multiples of bigger and bigger powers of 2.
there is something wrong there, it shouldn't be coming up with that many incorrect decimal places. There's a whole load of extra decimal places crept in, that seem to be more than the usual round off error.
sorry changed my mind, it is round off, you don't get it with all values e.g 7.59, but it's just the way it is, you have to do a final rounding yourself.
Or you could use the decimal type
decimal packagecharge = 7.56M;
decimal cnt = 3M;
decimal percost = packagecharge / cnt;
decimal diff = packagecharge - (percost * cnt);
Or you could use the decimal type
decimal packagecharge = 7.56M;
decimal cnt = 3M;
decimal percost = packagecharge / cnt;
decimal diff = packagecharge - (percost * cnt);
It is interesting to note that 756 is 4*7*27 ie; 3 cubed by 2 squared by 7. When you start dividing it by 10 to get 75.6 and then again by 10 to get 7.56, ie: by 5²2², the 2² cancels out but the 5² does not. The effect in floating point is that the last remaining digit in the inexact representation is a 5, which is what the questioner got.
deighton's decimal representation will work, since the internal representation to base 10 contains 5's and 2's, so dividing the number by 5's or 2's is not a problem, nor once by 7, or three times by 3, but by any other prime the same representational problems will exist.
There is an open source package (I've no idea whether it will work in a C# environment) to handle such rational calculations maintaining an arbitary precision. It's called BigRat - for Big Rational - which is also interesting and brings me full circle.
deighton's decimal representation will work, since the internal representation to base 10 contains 5's and 2's, so dividing the number by 5's or 2's is not a problem, nor once by 7, or three times by 3, but by any other prime the same representational problems will exist.
There is an open source package (I've no idea whether it will work in a C# environment) to handle such rational calculations maintaining an arbitary precision. It's called BigRat - for Big Rational - which is also interesting and brings me full circle.
ASKER
in this scenerio i ended up getting permission from accounting to round at a certain point, however this was enlightening.