Tom Farrar

asked on

# Simple Excel Calculation..

Why doesn't one of these formulas in attached spreadsheet columns B,C,D,E give me an answer of $.75? Not sure I understand why the $.750000000000002... What am I missing? Thanks.

EE.xlsx

EE.xlsx

Adjust your cell formatting to only display the .75

ASKER

It is not just formatting. I need the number to be exactly $.75

Try =ROUND(B2-B3,2) etc.

Its all to do with Excel's built in level of accuracy; It will only calculate to 15 significant places.

However, that doesn't help explain why two hard-coded (as opposed to calculated) numbers that are only 3/4 significant figures (2 before decimal and 2 after decimal) subsequently create a 15th significant figure, the 2 at the very end.

However, that doesn't help explain why two hard-coded (as opposed to calculated) numbers that are only 3/4 significant figures (2 before decimal and 2 after decimal) subsequently create a 15th significant figure, the 2 at the very end.

Try:

`=ROUNDDOWN(B2-B3,2)`

ASKER CERTIFIED SOLUTION

membership

Create an account to see this answer

Signing up is free. No credit card required.

Why doesn't one of these formulas in attached spreadsheet columns B,C,D,E give me an answer of $.75? Not sure I understand why the $.750000000000002... What am I missing? Thanks.

select the cells and use the number format

EE_currrency.xlsx
**Currency**―problem solved
The inaccuracy is caused by underlying data type which is "double" for the standard number. Double uses binary representation in 64 bits split between mantissa and exponent and the important thing is not all decimal numbers do have exact conversion to binary representation. Some (for us) simple numbers containing fractional part are converted to periodic binary number so certain rounding must happen.

Currency data type uses big integer (64 bits) with emulated 4 decimals obviously. So the accuracy is ensured if the 4 decimal places are enough. I am not sure whether Excel uses currency data type or just rounds the values.

Many programming languages and database engines (and some CPUs) use decimal data type which calculates and stores the decimal representation of the number without any conversion. Such data type cannot loose accuracy the same way as double. Of course calculations are much slower when used CPU does not support decimal data type.

Update: Sorry, the first paragraph was explained by Norie already.

Currency data type uses big integer (64 bits) with emulated 4 decimals obviously. So the accuracy is ensured if the 4 decimal places are enough. I am not sure whether Excel uses currency data type or just rounds the values.

Many programming languages and database engines (and some CPUs) use decimal data type which calculates and stores the decimal representation of the number without any conversion. Such data type cannot loose accuracy the same way as double. Of course calculations are much slower when used CPU does not support decimal data type.

Update: Sorry, the first paragraph was explained by Norie already.

ASKER

There were good thoughts by all, but Norie was first and most concise. Thank you all.