problem with float datatype and rounding and subnormal numbers

Hi,

I am working with a SQL database that returns some amount. The data types for amount column is float and rounding is alredy applied in the calculation.

The problem that I have at the moment is that I need to display all the record that have their amount as:

tblX.ColAmount > 0

the problem is that the ColAmount contains values such as: 0.000123-e12

I am not sure what this exactly does but the values are not returning anymore.

the questions are:
what is exactly this value 0.000123-e12 is and why they are stored in this way in the database ?

How  should I define this comparison tblX.ColAmount > 0 to get missing records displayed?


Thanks in advance

shmzAsked:
Who is Participating?
 
cyberkiwiCommented:
colAmounts > 0 should normally still return all of them.

See example here, all accounted for.

with tblX(id, colAmount) as (
select convert(int,234), convert(float,1.23E-16) union all
select 235,0.001 union all
select 236,1.24E-20 union all
select 237,1.55E-17)

select * from tblX where colAmount > 0
0
 
Patrick MatthewsCommented:
That is a very tiny, but non-zero number: 0.000123 times 10 ^ -12.Since it is greater than zero, it is getting returned by your query.In any event, you should try avoiding float if you possibly can, as it is an inherently imprecise data type.  Use decimal/numeric instead, which allows you to specify how many decimal places to carry, or use money (which uses 4 decimal places).
0
 
Ephraim WangoyaCommented:
Your rounding is resulting to zero thats why you get no records

if  tblX.ColAmountis 0.000123-e12
round(tblX.ColAmount, 3) -> 0
round(tblX.ColAmount, 4) -> 1
0
Cloud Class® Course: Amazon Web Services - Basic

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

 
cyberkiwiCommented:
The data types for amount column is float and rounding is already applied in the calculation.
How  should I define this comparison tblX.ColAmount > 0 to get missing records displayed?

Don't apply the rounding.
I think your number is  select 0.000123e-12  i.e. e-12, not -e12 right?

You need to round to 16 to even see any value from such a tiny number.

Round(tblX.ColAmount, 16)
0
 
shmzAuthor Commented:
data are already imported from another database with rounding already applied.
I am just displaying data in a report.
0
 
cyberkiwiCommented:
Can you elaborate on what you mean by this?

"I am not sure what this exactly does but the values are not returning anymore."

Do you expect some values with value of 0, that are now not showing anymore?
If the numbers are already rounded, where do you see this "0.000123e-12" ?
0
 
shmzAuthor Commented:
okay:

ID                        ColAmount
--------              -----------------
234                    0.000123e-12
235                     0
236                    0.000124e-16
237                    0.000155e-13

I search in the query for any colAmounts > 0.0001 and as a result only record with Id 235 returns while
I like to see all above records...
0
 
shmzAuthor Commented:
corecting my previous message:

ID                        ColAmount
--------              -----------------
234                    0.000123e-12
235                    0.001
236                    0.000124e-16
237                    0.000155e-13

I search in the query for any colAmounts > 0.0001 and as a result only record with Id 235 returns while
I like to see all above records...
0
 
cyberkiwiCommented:
To give you an idea of how big (small) the numbers are

The following table should tell you why > 0.0001 only returns one match.
ID	ColAmount  Actual
234	1.23E-16   0.0000000000000001230000
235	0.001      0.0010000000000000000000
236	1.24E-20   0.0000000000000000000124
237	1.55E-17   0.0000000000000000155000

Open in new window

0
 
shmzAuthor Commented:
good answer
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.