Solved

How to set double precision?

Posted on 2008-06-21
5
6,994 Views
Last Modified: 2013-12-16
My question is fairly simple. I just write a code that multiples to doubles say 3.13 and 3.13 and as a result I get 9.7968999999999991 instead of 9.7969. Why do I get those unwanted extra digits as 0.000099999991 etc. These extra digits propagate errors in the following calculations.

Could you suggest a solution pls!
0
Comment
Question by:Lexiks
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 

Expert Comment

by:jrshedden
ID: 21837310
Hi,
See if your can use BCD (binary coded decimal) types instead of doubles. Doubles calculate in radix 2 (binary) which doesn't always convert to/from radix 10 (decimal) exactly.

Jim
0
 
LVL 6

Expert Comment

by:psychic_zero
ID: 21837314
It's a system bug, you can use ToString function to set the decimal number like this
double number = 9.7968999999999991;
 
Console.WriteLine(number.ToString("#0.0000"));

Open in new window

0
 

Expert Comment

by:jrshedden
ID: 21837335
Hi,
I had a chance to look it up in the C# compiler. Have a look at "Decimal Structure" in the compiler help. The example should be helpful.
Jim
0
 
LVL 7

Accepted Solution

by:
photowhiz earned 125 total points
ID: 21839343
It is not a "bug", it is a fundamental property of the way computers represent fractions.  If you want to maintain a fixed number of decimal places, use the decimal type. Note that decimal types cannot represent fractions exactly either.

For more information, read http://docs.sun.com/source/806-3568/ncg_goldberg.html
0
 

Author Comment

by:Lexiks
ID: 21840097
Yeah photowhiz thats right. I myself kept searching and found the fact that you are saying. Ok thx all...
0

Featured Post

SharePoint Admin?

Enable Your Employees To Focus On The Core With Intuitive Onscreen Guidance That is With You At The Moment of Need.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

IP addresses can be stored in a database in any of several ways.  These ways may vary based on the volume of the data.  I was dealing with quite a large amount of data for user authentication purpose, and needed a way to minimize the storage.   …
Summary Displaying images in RichTextBox is a common requirement with limited solutions available. Pasting through clipboard or embedding into RTF content only support static images.  This article describes how to insert Windows control objects int…
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…
Visualize your data even better in Access queries. Given a date and a value, this lesson shows how to compare that value with the previous value, calculate the difference, and display a circle if the value is the same, an up triangle if it increased…

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question