# Comparing decimal values in classic asp is not producing correct result

I have a pretty straightforward comparison of two values that is not giving the intended results.

The page calculates the total qty of a sale of a part from history and compares that number to what the system thinks the total qty sold is.

Basically, I have an On Hand quantity of a part.  I calculate the entire part's sales and purchase orders to get with what the system should be at, based on history, and compare the two.

Recently, we decided to change the qty field from INT to FLOAT (sql 2014 back end) to accommodate for some manufacturing measurements.

So, as of now, I have a page that has this comparison:

if cDbl(rs("CalculatedShelfOH")) <> cDbl(rs("CardOH")) then
do stuff
end if

The CalculatdShelfOH is 2.126 and the CardOH is 2.126.  But, the code is "doing the stuff" even though they match.

What am I missing / doing wrong?
###### Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Analyst Assistant Commented:
It could be a problem with floating point precision/error/arithmetic, try rounding.

``````If Roundl(rs("CalculatedShelfOH"),3) <> Round(rs("CardOH"),3) Then
``````
0
Web Ninja at largeCommented:
i agree, it's probably a rounding issue. If you don't always know how many decimal places are going to be present, you could always convert the values to a string and do the compare:

if cStr(rs("CalculatedShelfOH")) <> cStr(rs("CardOH")) then
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Director, Information SystemsCommented:
From Microsoft Documentation:
"When you work with floating-point numbers, remember that they do not always have a precise representation in memory. This could lead to unexpected results from certain operations, such as value comparison and the Mod operator. For more information, see Troubleshooting Data Types."
0
Author Commented:
Thanks for the suggestions.  I will go with the cStr function as I will have many pages to update, and I just don't know how many decimals will be present on each function.
0
###### It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
ASP

From novice to tech pro — start learning today.