Solved

Bitwise comparison overflow error message

Posted on 2001-07-13
10
276 Views
Last Modified: 2008-02-01
Hi,
   I got an "Overflow" error message on VB and I don't know how to correct it.

My code is:

Const cstBinMskMaskProfLinteau as double = 30064771072#
Const cstBinMskProfLinteauNull as double=0
Const cstBinMskProfLinteau4 As Double = 4294967296
Dim Id_Fin_Tole as Double

Id_Fin_Tole=56754252

Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

  Case cstBinMskProfLinteauNull
     
    'some code here...    
     
  Case cstBinMskProfLinteau4
       
    'some code here...    
     
Note that VB had add a # after my first constant definition.  What does it meens???

The error is cause by the bitwise comparison itself((Id_Fin_Tole And cstBinMskMaskProfLinteau) return an overflow error message)

I use this line to know if the bits cstBinMskMaskProfLinteau are true in the number Id_Fin_Tole

Tanks a lot
0
Comment
Question by:Eddy24
10 Comments
 
LVL 44

Expert Comment

by:Arthur_Wood
ID: 6280843
the # is a "flag" to mark that value as a DOUBLE

what line causes the error?
0
 
LVL 1

Accepted Solution

by:
ElvasLion earned 200 total points
ID: 6280883
why do you use this line
Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

My problem is why do u use the AND operator that operator is used to do logical or bitwise operations

EX:

dim A as long
dim B as long
dim C as long
dim D as long
dim Result as long

A=10
B=8
C=6
D=Null

Result = A > B And B > C   ' Returns True.
Result = B > A And B > C   ' Returns False.
Result = A > B And B > D   ' Returns Null.
Result = A And B   ' Returns 8 (bitwise comparison).
0
 
LVL 4

Expert Comment

by:VincentLawlor
ID: 6280904
Checked it out it's on the AND operation

The two values are to large to AND together

Vin.
0
Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 
LVL 22

Expert Comment

by:rspahitz
ID: 6281116
First off, usually AND operations are performed on integers, so these should probably be defined as long.

Secondly, bit-wise operations have a limit, which used to be 16 bits (and may still be.)  By using a Double, you are attempting to bit-and on more bits than it can handle.

One option is to break the pieces up and then try to bit-and them:

Replace Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

 with

Temp1A = Id_Fin_Tole mod 65536 ' remove upper bits
Temp2A = int(Id_Fin_Tole /65536) ' remove lower bits
Temp1B = cstBinMskMaskProfLinteau mod 65536
Temp2B = int(cstBinMskMaskProfLinteau / 65536)

select case (Temp1A and Temp1B) ' check upper bits

...
Select Case (Temp2A and Temp2B) ' check upper bits
0
 
LVL 44

Expert Comment

by:Arthur_Wood
ID: 6282031
What EXACTLY are you trying to accomplish with the AND operator in the Select CASE statement?
0
 
LVL 5

Expert Comment

by:KDivad
ID: 6282513
<< ... a limit, which used to be 16 bits ... >>

Not anymore. VB's limit with Longs is 31 bits since the 32nd bit is "stolen" to make a signed long. In that case, you can check it's pos/neg status to get the 32nd bit. Keep in mind that the bits run 0-31, so the 32nd bit is number 31. I would assume that Doubles/etc. would allow higher bit numbers...


Anyway, Id_Fin_Tole is probably cast as a Long and cstBinMskMaskProfLinteau is a Double. VB may be attempting to cast the entire operation as a Long which won't work. Try this instead (I haven't tested!):

CDbl(Id_Fin_Tole) And cstBinMskMaskProfLinteau
0
 
LVL 75

Expert Comment

by:Anthony Perkins
ID: 6283724
Actually the defintion of And/Or for bitwise operations asks for numeric expressions not just integers. So the following is quite legal:

Dim x As Double
Dim y As Double

x = 10
y = 200

Debug.Print x And y

Though why you would use it is beyond me. Not to mention the inherent risk of rounding problems when using doubles.

In any case, as has been pointed out already, the problem has to do with an overflow with the result of the bitwise comparison.

0
 
LVL 5

Expert Comment

by:KDivad
ID: 6284639
<<... Id_Fin_Tole is probably cast as a Long ...>>

My mistake. I missed the Dim statement for Id_Fin_Tole. What I explained can happen with mixed data types, though...
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 7153095
Hi Eddy24,
It appears that you have forgotten this question. I will ask Community Support to close it unless you finalize it within 7 days. I will ask a Community Support Moderator to:

    Accept ElvasLion's comment(s) as an answer.
    *** first to mention the AND

Eddy24, if you think your question was not answered at all or if you need help, just post a new comment here; Community Support will help you.  DO NOT accept this comment as an answer.

EXPERTS: If you disagree with that recommendation, please post an explanatory comment.
==========
DanRollins -- EE database cleanup volunteer
0
 
LVL 1

Expert Comment

by:Computer101
ID: 7182156
Comment from expert accepted as answer

Computer101
E-E Moderator
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Background What I'm presenting in this article is the result of 2 conditions in my work area: We have a SQL Server production environment but no development or test environment; andWe have an MS Access front end using tables in SQL Server but we a…
Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
Get people started with the process of using Access VBA to control Excel using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Excel. Using automation, an Access application can laun…
Get people started with the utilization of class modules. Class modules can be a powerful tool in Microsoft Access. They allow you to create self-contained objects that encapsulate functionality. They can easily hide the complexity of a process from…

791 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question