asked on Simple Excel Calculation..
Why doesn't one of these formulas in attached spreadsheet columns B,C,D,E give me an answer of $.75? Not sure I understand why the $.750000000000002... What am I missing? Thanks.
EE.xlsx
EE.xlsx
Microsoft OfficeMicrosoft ExcelMicrosoft ApplicationsSpreadsheets
Last Comment
Tom Farrar
Adjust your cell formatting to only display the .75
ASKER
It is not just formatting. I need the number to be exactly $.75
Try =ROUND(B2-B3,2) etc.
Its all to do with Excel's built in level of accuracy; It will only calculate to 15 significant places.
However, that doesn't help explain why two hard-coded (as opposed to calculated) numbers that are only 3/4 significant figures (2 before decimal and 2 after decimal) subsequently create a 15th significant figure, the 2 at the very end.
However, that doesn't help explain why two hard-coded (as opposed to calculated) numbers that are only 3/4 significant figures (2 before decimal and 2 after decimal) subsequently create a 15th significant figure, the 2 at the very end.
Try:
=ROUNDDOWN(B2-B3,2)
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Why doesn't one of these formulas in attached spreadsheet columns B,C,D,E give me an answer of $.75? Not sure I understand why the $.750000000000002... What am I missing? Thanks.
select the cells and use the number format Currency―problem solved
EE_currrency.xlsx
The inaccuracy is caused by underlying data type which is "double" for the standard number. Double uses binary representation in 64 bits split between mantissa and exponent and the important thing is not all decimal numbers do have exact conversion to binary representation. Some (for us) simple numbers containing fractional part are converted to periodic binary number so certain rounding must happen.
Currency data type uses big integer (64 bits) with emulated 4 decimals obviously. So the accuracy is ensured if the 4 decimal places are enough. I am not sure whether Excel uses currency data type or just rounds the values.
Many programming languages and database engines (and some CPUs) use decimal data type which calculates and stores the decimal representation of the number without any conversion. Such data type cannot loose accuracy the same way as double. Of course calculations are much slower when used CPU does not support decimal data type.
Update: Sorry, the first paragraph was explained by Norie already.
Currency data type uses big integer (64 bits) with emulated 4 decimals obviously. So the accuracy is ensured if the 4 decimal places are enough. I am not sure whether Excel uses currency data type or just rounds the values.
Many programming languages and database engines (and some CPUs) use decimal data type which calculates and stores the decimal representation of the number without any conversion. Such data type cannot loose accuracy the same way as double. Of course calculations are much slower when used CPU does not support decimal data type.
Update: Sorry, the first paragraph was explained by Norie already.
ASKER
There were good thoughts by all, but Norie was first and most concise. Thank you all.