OverFlow Error

I need to store decimal point values in a variable, the range of values is 19.999999 to 0.999999...

The problem is what type do I declare my variable as because....

Integer and Long no good as it rounds decimal places...

Decimal has a range of "For numbers with 28 decimal places, the range is
+/-7.9228162514264337593543950335. The smallest possible non-zero number is 0.0000000000000000000000000001."

Single has a range of "-3.402823E38 to -1.401298E-45 for negative values; 1.401298E-45 to 3.402823E38 for positive values."
 
Variant is the same as "Same range as Double for numerics."  

Thanks Ian

LVL 1
ISCAsked:
Who is Participating?
 
RomModConnect With a Mentor Commented:
Question closed - 500 points refunded.

Best regards,
RomMod
Experts Exchange
Community Support Moderator
0
 
GrahamSkanRetiredCommented:
You can store the numbers 19.999999 to 0.999999... in doubles. How many significant digits (how accurate) do you need?
0
 
ISCAuthor Commented:
At least 2 significant digits and 6 decimal places...

I tried this but I get an overflow error...


Ian
0
Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
ISCAuthor Commented:
The calculation causing the problem ss...

   Dim k As Double
       
    k = m_objAccrualRecord.m_intNumberOfDays * 100 * l_dblDailyAccrualTotal

This caused an overflow, however if I change the code to...

  Dim k As Double
       
        k = CDbl(m_objAccrualRecord.m_intNumberOfDays) * CDbl(100) * l_dblDailyAccrualTotal

It works....!
0
 
GrahamSkanRetiredCommented:
There is a buglet in VB. The calculation can start off assuming (for speed) that it is using integers, only to be confounded later on by a number too big to fit into its temporary location.
0
 
ISCAuthor Commented:
Thanks I did not know that...!
0
 
EDDYKTCommented:
how about


k = l_dblDailyAccrualTotal * 100.0 * m_objAccrualRecord.m_intNumberOfDays
0
 
EDDYKTCommented:
or


k = m_objAccrualRecord.m_intNumberOfDays * 100.0 * l_dblDailyAccrualTotal
0
 
ISCAuthor Commented:
My solution was to use CDbl( ) in the formula this worked and stopped the overflow error...!

 Dim k As Double
       
        k = CDbl(m_objAccrualRecord.m_intNumberOfDays) * CDbl(100) * l_dblDailyAccrualTotal
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.