Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Bitwise comparison overflow error message

Posted on 2001-07-13
10
Medium Priority
?
282 Views
Last Modified: 2008-02-01
Hi,
   I got an "Overflow" error message on VB and I don't know how to correct it.

My code is:

Const cstBinMskMaskProfLinteau as double = 30064771072#
Const cstBinMskProfLinteauNull as double=0
Const cstBinMskProfLinteau4 As Double = 4294967296
Dim Id_Fin_Tole as Double

Id_Fin_Tole=56754252

Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

  Case cstBinMskProfLinteauNull
     
    'some code here...    
     
  Case cstBinMskProfLinteau4
       
    'some code here...    
     
Note that VB had add a # after my first constant definition.  What does it meens???

The error is cause by the bitwise comparison itself((Id_Fin_Tole And cstBinMskMaskProfLinteau) return an overflow error message)

I use this line to know if the bits cstBinMskMaskProfLinteau are true in the number Id_Fin_Tole

Tanks a lot
0
Comment
Question by:Eddy24
10 Comments
 
LVL 44

Expert Comment

by:Arthur_Wood
ID: 6280843
the # is a "flag" to mark that value as a DOUBLE

what line causes the error?
0
 
LVL 1

Accepted Solution

by:
ElvasLion earned 800 total points
ID: 6280883
why do you use this line
Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

My problem is why do u use the AND operator that operator is used to do logical or bitwise operations

EX:

dim A as long
dim B as long
dim C as long
dim D as long
dim Result as long

A=10
B=8
C=6
D=Null

Result = A > B And B > C   ' Returns True.
Result = B > A And B > C   ' Returns False.
Result = A > B And B > D   ' Returns Null.
Result = A And B   ' Returns 8 (bitwise comparison).
0
 
LVL 4

Expert Comment

by:VincentLawlor
ID: 6280904
Checked it out it's on the AND operation

The two values are to large to AND together

Vin.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 22

Expert Comment

by:rspahitz
ID: 6281116
First off, usually AND operations are performed on integers, so these should probably be defined as long.

Secondly, bit-wise operations have a limit, which used to be 16 bits (and may still be.)  By using a Double, you are attempting to bit-and on more bits than it can handle.

One option is to break the pieces up and then try to bit-and them:

Replace Select Case (Id_Fin_Tole And cstBinMskMaskProfLinteau)

 with

Temp1A = Id_Fin_Tole mod 65536 ' remove upper bits
Temp2A = int(Id_Fin_Tole /65536) ' remove lower bits
Temp1B = cstBinMskMaskProfLinteau mod 65536
Temp2B = int(cstBinMskMaskProfLinteau / 65536)

select case (Temp1A and Temp1B) ' check upper bits

...
Select Case (Temp2A and Temp2B) ' check upper bits
0
 
LVL 44

Expert Comment

by:Arthur_Wood
ID: 6282031
What EXACTLY are you trying to accomplish with the AND operator in the Select CASE statement?
0
 
LVL 5

Expert Comment

by:KDivad
ID: 6282513
<< ... a limit, which used to be 16 bits ... >>

Not anymore. VB's limit with Longs is 31 bits since the 32nd bit is "stolen" to make a signed long. In that case, you can check it's pos/neg status to get the 32nd bit. Keep in mind that the bits run 0-31, so the 32nd bit is number 31. I would assume that Doubles/etc. would allow higher bit numbers...


Anyway, Id_Fin_Tole is probably cast as a Long and cstBinMskMaskProfLinteau is a Double. VB may be attempting to cast the entire operation as a Long which won't work. Try this instead (I haven't tested!):

CDbl(Id_Fin_Tole) And cstBinMskMaskProfLinteau
0
 
LVL 75

Expert Comment

by:Anthony Perkins
ID: 6283724
Actually the defintion of And/Or for bitwise operations asks for numeric expressions not just integers. So the following is quite legal:

Dim x As Double
Dim y As Double

x = 10
y = 200

Debug.Print x And y

Though why you would use it is beyond me. Not to mention the inherent risk of rounding problems when using doubles.

In any case, as has been pointed out already, the problem has to do with an overflow with the result of the bitwise comparison.

0
 
LVL 5

Expert Comment

by:KDivad
ID: 6284639
<<... Id_Fin_Tole is probably cast as a Long ...>>

My mistake. I missed the Dim statement for Id_Fin_Tole. What I explained can happen with mixed data types, though...
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 7153095
Hi Eddy24,
It appears that you have forgotten this question. I will ask Community Support to close it unless you finalize it within 7 days. I will ask a Community Support Moderator to:

    Accept ElvasLion's comment(s) as an answer.
    *** first to mention the AND

Eddy24, if you think your question was not answered at all or if you need help, just post a new comment here; Community Support will help you.  DO NOT accept this comment as an answer.

EXPERTS: If you disagree with that recommendation, please post an explanatory comment.
==========
DanRollins -- EE database cleanup volunteer
0
 
LVL 1

Expert Comment

by:Computer101
ID: 7182156
Comment from expert accepted as answer

Computer101
E-E Moderator
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction While answering a recent question (http://www.experts-exchange.com/Q_27402310.html) in the VB classic zone, I wrote some VB code in the (Office) VBA environment, rather than fire up my older PC.  I didn't post completely correct code o…
Introduction While answering a recent question about filtering a custom class collection, I realized that this could be accomplished with very little code by using the ScriptControl (SC) library.  This article will introduce you to the SC library a…
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…
Suggested Courses

926 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question