Solved

How to set double precision?

Posted on 2008-06-21
5
6,993 Views
Last Modified: 2013-12-16
My question is fairly simple. I just write a code that multiples to doubles say 3.13 and 3.13 and as a result I get 9.7968999999999991 instead of 9.7969. Why do I get those unwanted extra digits as 0.000099999991 etc. These extra digits propagate errors in the following calculations.

Could you suggest a solution pls!
0
Comment
Question by:Lexiks
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 

Expert Comment

by:jrshedden
ID: 21837310
Hi,
See if your can use BCD (binary coded decimal) types instead of doubles. Doubles calculate in radix 2 (binary) which doesn't always convert to/from radix 10 (decimal) exactly.

Jim
0
 
LVL 6

Expert Comment

by:psychic_zero
ID: 21837314
It's a system bug, you can use ToString function to set the decimal number like this
double number = 9.7968999999999991;
 
Console.WriteLine(number.ToString("#0.0000"));

Open in new window

0
 

Expert Comment

by:jrshedden
ID: 21837335
Hi,
I had a chance to look it up in the C# compiler. Have a look at "Decimal Structure" in the compiler help. The example should be helpful.
Jim
0
 
LVL 7

Accepted Solution

by:
photowhiz earned 125 total points
ID: 21839343
It is not a "bug", it is a fundamental property of the way computers represent fractions.  If you want to maintain a fixed number of decimal places, use the decimal type. Note that decimal types cannot represent fractions exactly either.

For more information, read http://docs.sun.com/source/806-3568/ncg_goldberg.html
0
 

Author Comment

by:Lexiks
ID: 21840097
Yeah photowhiz thats right. I myself kept searching and found the fact that you are saying. Ok thx all...
0

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Welcome my friends to the second instalment and follow-up to our Minify and Concatenate Your Scripts and Stylesheets (http://www.experts-exchange.com/Programming/Languages/.NET/ASP.NET/A_4334-Minify-and-Concatenate-Your-Scripts-and-Stylesheets.html)…
This article shows how to deploy dynamic backgrounds to computers depending on the aspect ratio of display
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question