Solved

Bitwise comparison overflow error message

Posted on 2001-07-13
10
273 Views
Last Modified: 2008-02-01
Hi,
   I got an "Overflow" error message on VB and I don't know how to correct it.

My code is:

Const cstBinMskMaskProfLinteau as double = 30064771072#
Const cstBinMskProfLinteauNull as double=0
Const cstBinMskProfLinteau4 As Double = 4294967296
Dim Id_Fin_Tole as Double

Id_Fin_Tole=56754252

Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

  Case cstBinMskProfLinteauNull
     
    'some code here...    
     
  Case cstBinMskProfLinteau4
       
    'some code here...    
     
Note that VB had add a # after my first constant definition.  What does it meens???

The error is cause by the bitwise comparison itself((Id_Fin_Tole And cstBinMskMaskProfLinteau) return an overflow error message)

I use this line to know if the bits cstBinMskMaskProfLinteau are true in the number Id_Fin_Tole

Tanks a lot
0
Comment
Question by:Eddy24
10 Comments
 
LVL 44

Expert Comment

by:Arthur_Wood
ID: 6280843
the # is a "flag" to mark that value as a DOUBLE

what line causes the error?
0
 
LVL 1

Accepted Solution

by:
ElvasLion earned 200 total points
ID: 6280883
why do you use this line
Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

My problem is why do u use the AND operator that operator is used to do logical or bitwise operations

EX:

dim A as long
dim B as long
dim C as long
dim D as long
dim Result as long

A=10
B=8
C=6
D=Null

Result = A > B And B > C   ' Returns True.
Result = B > A And B > C   ' Returns False.
Result = A > B And B > D   ' Returns Null.
Result = A And B   ' Returns 8 (bitwise comparison).
0
 
LVL 4

Expert Comment

by:VincentLawlor
ID: 6280904
Checked it out it's on the AND operation

The two values are to large to AND together

Vin.
0
 
LVL 22

Expert Comment

by:rspahitz
ID: 6281116
First off, usually AND operations are performed on integers, so these should probably be defined as long.

Secondly, bit-wise operations have a limit, which used to be 16 bits (and may still be.)  By using a Double, you are attempting to bit-and on more bits than it can handle.

One option is to break the pieces up and then try to bit-and them:

Replace Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

 with

Temp1A = Id_Fin_Tole mod 65536 ' remove upper bits
Temp2A = int(Id_Fin_Tole /65536) ' remove lower bits
Temp1B = cstBinMskMaskProfLinteau mod 65536
Temp2B = int(cstBinMskMaskProfLinteau / 65536)

select case (Temp1A and Temp1B) ' check upper bits

...
Select Case (Temp2A and Temp2B) ' check upper bits
0
 
LVL 44

Expert Comment

by:Arthur_Wood
ID: 6282031
What EXACTLY are you trying to accomplish with the AND operator in the Select CASE statement?
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 5

Expert Comment

by:KDivad
ID: 6282513
<< ... a limit, which used to be 16 bits ... >>

Not anymore. VB's limit with Longs is 31 bits since the 32nd bit is "stolen" to make a signed long. In that case, you can check it's pos/neg status to get the 32nd bit. Keep in mind that the bits run 0-31, so the 32nd bit is number 31. I would assume that Doubles/etc. would allow higher bit numbers...


Anyway, Id_Fin_Tole is probably cast as a Long and cstBinMskMaskProfLinteau is a Double. VB may be attempting to cast the entire operation as a Long which won't work. Try this instead (I haven't tested!):

CDbl(Id_Fin_Tole) And cstBinMskMaskProfLinteau
0
 
LVL 75

Expert Comment

by:Anthony Perkins
ID: 6283724
Actually the defintion of And/Or for bitwise operations asks for numeric expressions not just integers. So the following is quite legal:

Dim x As Double
Dim y As Double

x = 10
y = 200

Debug.Print x And y

Though why you would use it is beyond me. Not to mention the inherent risk of rounding problems when using doubles.

In any case, as has been pointed out already, the problem has to do with an overflow with the result of the bitwise comparison.

0
 
LVL 5

Expert Comment

by:KDivad
ID: 6284639
<<... Id_Fin_Tole is probably cast as a Long ...>>

My mistake. I missed the Dim statement for Id_Fin_Tole. What I explained can happen with mixed data types, though...
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 7153095
Hi Eddy24,
It appears that you have forgotten this question. I will ask Community Support to close it unless you finalize it within 7 days. I will ask a Community Support Moderator to:

    Accept ElvasLion's comment(s) as an answer.
    *** first to mention the AND

Eddy24, if you think your question was not answered at all or if you need help, just post a new comment here; Community Support will help you.  DO NOT accept this comment as an answer.

EXPERTS: If you disagree with that recommendation, please post an explanatory comment.
==========
DanRollins -- EE database cleanup volunteer
0
 
LVL 1

Expert Comment

by:Computer101
ID: 7182156
Comment from expert accepted as answer

Computer101
E-E Moderator
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Article by: Martin
Here are a few simple, working, games that you can use as-is or as the basis for your own games. Tic-Tac-Toe This is one of the simplest of all games.   The game allows for a choice of who goes first and keeps track of the number of wins for…
Enums (shorthand for ‘enumerations’) are not often used by programmers but they can be quite valuable when they are.  What are they? An Enum is just a type of variable like a string or an Integer, but in this case one that you create that contains…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Get people started with the utilization of class modules. Class modules can be a powerful tool in Microsoft Access. They allow you to create self-contained objects that encapsulate functionality. They can easily hide the complexity of a process from…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now