Solved

strange overflow problem!

Posted on 2000-04-21
9
174 Views
Last Modified: 2010-05-02
puzzling thigs in VB I am trying to figure out:

      18 * 60000 gives me back  
1080000

while   40 *1000 overflows!

Why?!!!
0
Comment
Question by:senardji
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +3
9 Comments
 
LVL 28

Expert Comment

by:AzraSound
ID: 2738459
what data type have you declared to hold these values???
0
 
LVL 7

Accepted Solution

by:
Vbmaster earned 50 total points
ID: 2738481
VB interprets the two variables as integers, and two integers gives another integer. The first expression has one variable (60000) that is a Long variable and one integer * one long => one long.

You can do something like 40 * CLng(1000) to make sure the 1000 is interpreted as a Long variable and therefore the whole expresion will be of Long type.
0
 
LVL 7

Expert Comment

by:Vbmaster
ID: 2738488
Another note.. A integer can only hold values between -32768 and +32767 (40000 is therefore too big).
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Expert Comment

by:yang051597
ID: 2738511
if you are using variants or you have not defined the types of variables you are using, the compiler will convert your variable to the lowest datatype that can satisfy to represent the value.

in your example:
  40 * 1000 = overflow

  40 will be considered an integer
1000 will also be considered an integer because integers range from -32,768 to 32,767. So in effect you are multiplying two integers and the result should also be an integer. Since the result is 40,000 which is bigger than the max integer value, we have an overflow.


18 * 60000
the compiler will treat the 60000 as a Long since it can represent the value.
In effect, we are multipliying longs which will also give us a Long.
the range of longs is -2,147,483,648 to 2,147,483,647. So the result is within range.

To avoid this kind of problem, make sure that you define your variables correctly.

0
 
LVL 5

Expert Comment

by:KDivad
ID: 2739439
yang, you should retract your answer, because vbmaster posted basically the same thing before you did.
0
 
LVL 12

Expert Comment

by:mark2150
ID: 2739866
Actually the easiest way to get this to work is put a # on the end of the numbers so that vb knows you're working with doubles instead of integers.

40# * 1000#

will NOT overflow. Sometimes dynamic typing helps, sometimes dynamic typing hurts.

You'ld think that VB would be smart enough to look at the *entire* equation and find the largest type and work the problem that way instead of hunting for the smallest type and generation stupid overflow errors.

If you code:

DIM Result as Double
Result = 40 * 1000

It'll still overflow even tho the receiving var is explicitly typed as a double. The right side is all integers and therefore is performed in integer math.

One really screwy thing that shows the effects of dynamic typing is:

DIM A as Double
DIM B as Double
DIM C as Double

A = 40
B = 1000
C = A * B

This will *still* overflow even though all of the vars are explicitly typed as Doubles. The assignments 'A=40' and 'B=1000' are both assigning integer values to Double Vars - and VB dutifully notices that only integers are needed and so A and B are dynamically retyped as Integers and the 'C = A * B' statement chokes.

So, the lesson is that you should make sure your numerical constants are typed the same as your vars. Using:

A = 40#
B = 1000#

will cause the compiler to treat the numerics as doubles (you can also enter 40.0 and 1000.0) and will prevent dynamic typing from biting you.

M
0
 

Expert Comment

by:yang051597
ID: 2742038
KDivad : Sorry about that. That's the price of not refreshing my browser.
0
 
LVL 5

Expert Comment

by:KDivad
ID: 2742169
yang, no problem. I've done it myself, so I always wait for the page to reload afte posting to see if it happened to me *again*. :-)
0
 

Author Comment

by:senardji
ID: 2743973
Thanks.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I’ve seen a number of people looking for examples of how to access web services from VB6.  I’ve been using a test harness I built in VB6 (using many resources I found online) that I use for small projects to work out how to communicate with web serv…
Article by: Martin
Here are a few simple, working, games that you can use as-is or as the basis for your own games. Tic-Tac-Toe This is one of the simplest of all games.   The game allows for a choice of who goes first and keeps track of the number of wins for…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…
Suggested Courses
Course of the Month8 days, left to enroll

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question