?
Solved

VB.NET Subtraction Problem

Posted on 2006-04-19
4
Medium Priority
?
522 Views
Last Modified: 2008-02-20
Hi,

I'm currenty using VS.NET 2003 .NET 1.1 and am experiencing a very strange problem.

        Dim x As Double, y As Double, z As Double

        x = 10.2
        y = 3
        z = x - y

At the end of this z is always  7.199999999999993

Where do all of these decimal places come from? I'm sure there's a very simple answer or I haven't got something set right, but I can't find it.

Thanks in advance,
Jon
0
Comment
Question by:HBPROCK
  • 2
  • 2
4 Comments
 
LVL 96

Expert Comment

by:Bob Learned
ID: 16490021
It is based on the precision for double.  If you don't need it, don't use it--use singles instead.

Dim x As Single, y As Single, z As Single

x = 10.2
y = 3
z = x - y

Bob
0
 

Author Comment

by:HBPROCK
ID: 16490205
I see your point Bob, but in certain circumstances (i.e my calculator) I will require a dynamic range of results.

Many more DP may be required, but smaller sums like this will still need to return an accurate result without seemingly erronious variation. Using rounding is not an option...

Could you explain to me where this variation from the math logic comes from? Being a VB6 developer, I've always been used to receiving the 7.2 expected from this calculation.

Many Thanks,
Jon
0
 
LVL 96

Accepted Solution

by:
Bob Learned earned 400 total points
ID: 16490256
http://msdn2.microsoft.com/en-us/library/x99xtshc(VS.80).aspx

<Quote>
Double precision numbers store an approximation of a real number.

When you work with floating-point numbers, keep in mind that they do not always have a precise representation in memory. This could lead to unexpected results from certain operations...
</Quote>

You only need as much precision from the result that you have with the input:

z = Math.Round(z + .005, 1)

Bob
0
 

Author Comment

by:HBPROCK
ID: 16490379
This is the information I needed...

Thanks!
Jon
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: jpaulino
XML Literals are a great way to handle XML files and the community doesn’t use it as much as it should.  An XML Literal is like a String (http://msdn.microsoft.com/en-us/library/system.string.aspx) Literal, only instead of starting and ending with w…
Introduction When many people think of the WebBrowser (http://msdn.microsoft.com/en-us/library/2te2y1x6%28v=VS.85%29.aspx) control, they immediately think of a control which allows the viewing and navigation of web pages. While this is true, it's a…
Integration Management Part 2
Look below the covers at a subform control , and the form that is inside it. Explore properties and see how easy it is to aggregate, get statistics, and synchronize results for your data. A Microsoft Access subform is used to show relevant calcul…
Suggested Courses

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question