Solved

Double type for whole number?

Posted on 2003-11-25
7
357 Views
Last Modified: 2010-05-01
Hi Experts,

Got some data by whole number calculation, I need to store the data in an array and some variables. The data are bigger than the max number in Long type.
Do you think I can use Double type to store the whole number without lossing the acuracy? say, would it store 333333333 as 333333332.9999999?
If it lost some acuracy I will get problem because I need to compare such stored data.
0
Comment
Question by:dartnut
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 5

Accepted Solution

by:
mccainz2 earned 35 total points
ID: 9818902
Double stores up to 14 decimal places (dependant on how many places you have to the left of the decimal)

as for inaccuracy , if it starts life as a whole number you should be fine ...

try...

Dim x As Double
 x = 333333333
MsgBox CDbl(x / 3)

and for comparison try

Dim x, y As Double
x = 3
y = 3.00000000000001
MsgBox CBool(x = y)
0
 
LVL 7

Expert Comment

by:Z_Beeblebrox
ID: 9819278
If you are concerned about accuracy, you could use the Decimal type, which will store such numbers exactly, but with a large performance penalty.

Zaphod.
0
 
LVL 44

Assisted Solution

by:Arthur_Wood
Arthur_Wood earned 35 total points
ID: 9819841
just as a matter of note, the value 333333333 is NOT too large for a VB Long variable

Long Datatype has an UPPER limit of 2147483647, so 333333333  is well below that limit.

AW
                                                     
0
 
LVL 41

Expert Comment

by:graye
ID: 9835024
Floats, doubles, and decimals are not really well suited for storing big integers... their designs allow for the number of significant digits (the mantissa) portion to actually shrink/grow by 1 digit depending the sign and exponents.

There are lots of "big integer" libraries out there...  VB.Net is a better platform for doing this kind of stuff.  If you're interested, I've got some links/source code for big integer libraries in both VB6 and VB.Net

Just curious... how'd you get the numbers anyway... (since you've got to have 'em stored in a variable somewhere before you can put 'em in an array)
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction While answering a recent question about filtering a custom class collection, I realized that this could be accomplished with very little code by using the ScriptControl (SC) library.  This article will introduce you to the SC library a…
Article by: Martin
Here are a few simple, working, games that you can use as-is or as the basis for your own games. Tic-Tac-Toe This is one of the simplest of all games.   The game allows for a choice of who goes first and keeps track of the number of wins for…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…

740 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question