Solved

# Bitwise comparison overflow error message

Posted on 2001-07-13
278 Views
Hi,
I got an "Overflow" error message on VB and I don't know how to correct it.

My code is:

Const cstBinMskMaskProfLinteau as double = 30064771072#
Const cstBinMskProfLinteauNull as double=0
Const cstBinMskProfLinteau4 As Double = 4294967296
Dim Id_Fin_Tole as Double

Id_Fin_Tole=56754252

Case cstBinMskProfLinteauNull

'some code here...

Case cstBinMskProfLinteau4

'some code here...

Note that VB had add a # after my first constant definition.  What does it meens???

The error is cause by the bitwise comparison itself((Id_Fin_Tole And cstBinMskMaskProfLinteau) return an overflow error message)

I use this line to know if the bits cstBinMskMaskProfLinteau are true in the number Id_Fin_Tole

Tanks a lot
0
Question by:Eddy24
[X]
###### Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

• Help others & share knowledge
• Earn cash & points

LVL 44

Expert Comment

ID: 6280843
the # is a "flag" to mark that value as a DOUBLE

what line causes the error?
0

LVL 1

Accepted Solution

ElvasLion earned 200 total points
ID: 6280883
why do you use this line

My problem is why do u use the AND operator that operator is used to do logical or bitwise operations

EX:

dim A as long
dim B as long
dim C as long
dim D as long
dim Result as long

A=10
B=8
C=6
D=Null

Result = A > B And B > C   ' Returns True.
Result = B > A And B > C   ' Returns False.
Result = A > B And B > D   ' Returns Null.
Result = A And B   ' Returns 8 (bitwise comparison).
0

LVL 4

Expert Comment

ID: 6280904
Checked it out it's on the AND operation

The two values are to large to AND together

Vin.
0

LVL 22

Expert Comment

ID: 6281116
First off, usually AND operations are performed on integers, so these should probably be defined as long.

Secondly, bit-wise operations have a limit, which used to be 16 bits (and may still be.)  By using a Double, you are attempting to bit-and on more bits than it can handle.

One option is to break the pieces up and then try to bit-and them:

Replace Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

with

Temp1A = Id_Fin_Tole mod 65536 ' remove upper bits
Temp2A = int(Id_Fin_Tole /65536) ' remove lower bits

select case (Temp1A and Temp1B) ' check upper bits

...
Select Case (Temp2A and Temp2B) ' check upper bits
0

LVL 44

Expert Comment

ID: 6282031
What EXACTLY are you trying to accomplish with the AND operator in the Select CASE statement?
0

LVL 5

Expert Comment

ID: 6282513
<< ... a limit, which used to be 16 bits ... >>

Not anymore. VB's limit with Longs is 31 bits since the 32nd bit is "stolen" to make a signed long. In that case, you can check it's pos/neg status to get the 32nd bit. Keep in mind that the bits run 0-31, so the 32nd bit is number 31. I would assume that Doubles/etc. would allow higher bit numbers...

Anyway, Id_Fin_Tole is probably cast as a Long and cstBinMskMaskProfLinteau is a Double. VB may be attempting to cast the entire operation as a Long which won't work. Try this instead (I haven't tested!):

0

LVL 75

Expert Comment

ID: 6283724
Actually the defintion of And/Or for bitwise operations asks for numeric expressions not just integers. So the following is quite legal:

Dim x As Double
Dim y As Double

x = 10
y = 200

Debug.Print x And y

Though why you would use it is beyond me. Not to mention the inherent risk of rounding problems when using doubles.

In any case, as has been pointed out already, the problem has to do with an overflow with the result of the bitwise comparison.

0

LVL 5

Expert Comment

ID: 6284639
<<... Id_Fin_Tole is probably cast as a Long ...>>

My mistake. I missed the Dim statement for Id_Fin_Tole. What I explained can happen with mixed data types, though...
0

LVL 49

Expert Comment

ID: 7153095
Hi Eddy24,
It appears that you have forgotten this question. I will ask Community Support to close it unless you finalize it within 7 days. I will ask a Community Support Moderator to:

Accept ElvasLion's comment(s) as an answer.
*** first to mention the AND

Eddy24, if you think your question was not answered at all or if you need help, just post a new comment here; Community Support will help you.  DO NOT accept this comment as an answer.

EXPERTS: If you disagree with that recommendation, please post an explanatory comment.
==========
DanRollins -- EE database cleanup volunteer
0

LVL 1

Expert Comment

ID: 7182156
Comment from expert accepted as answer

Computer101
E-E Moderator
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

### Suggested Solutions

Introduction I needed to skip over some file processing within a For...Next loop in some old production code and wished that VB (classic) had a statement that would drop down to the end of the current iteration, bypassing the statements that were c…
Here are a few simple, working, games that you can use as-is or as the basis for your own games. Tic-Tac-Toe This is one of the simplest of all games.   The game allows for a choice of who goes first and keeps track of the number of wins for…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…
###### Suggested Courses
Course of the Month1 day, 20 hours left to enroll