We help IT Professionals succeed at work.

Convert Float to Integer in Groovy gives rounding errors

Steve Wales
Steve Wales asked
on
I am having to grab some data from an Oracle database via Groovy script and dump it into a flat file to upload to a bank for EFT payments.

The data is stored in the database as NUMBER(15,2) - it represents a dollar value.

The bank expects the number to be written out as a string, without the decimal.

My thought was to read the value from the database ($528.10) and store it in a float - which seems to work.

In order to output the value as 0000052810 (value is 10 characters, assuming 8 digits for dollars and last 2 digits for cents) I thought I'd try this.

Float x1
x1 = 528.1
x2 = x1 * 100

Open in new window


However multiplying the float by 100 is giving me rounding errors.  

I added output of the two values:

println x1.toString()
println x2.toString()

Which gave me:

528.1
52809.99755859375

I could add: x3 = x2.round() to make the output 52810 - but since I'm dealing with financial information here, I'd rather get an exact conversion than rely on rounding.  Anyone able to offer me a clean solution to this ?
Comment
Watch Question

ozo
Most Valuable Expert 2014
Top Expert 2015
Commented:
528.1 is not representable exactly in binary floating point, so it is approximated by 528.0999755859375
If you are dealing with exact values, you can instead do computations on the exact integer value 52810 and only convert to approximate float values on output.

https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/02Numerics/Double/paper.pdf
Steve WalesSenior Database Administrator

Author

Commented:
From a coding point of view though, I'm starting with the floating point value of 528.10 - getting it to the integer value of 52810 is the problem ...
ozo
Most Valuable Expert 2014
Top Expert 2015
Commented:
There is no such floating point value as 528.10
If you start there, you are starting with an error.
If you happen to know that the erroneous float value in the database should actually be an integer value / 100 then you could round it to determine the true value.
Awarded 2010
Top Expert 2013
Commented:
Multiply it by 100 as a float and then round
x2 = (x1*100).round();
Awarded 2010
Top Expert 2013
Commented:
Although I definitely agree with Ozo. Making the number a float isn't really ideal. The underlying storage for the Oracle NUMBER data type is a decimal floating point number where Java/Groovy float is binary floating point. If you can extract the number as an integer (like 52810) and add the decimal point, it will be more precisely accurate.

However, if you are only working with cents, rounding will give you the correct answer every time.
ozo
Most Valuable Expert 2014
Top Expert 2015
Commented:
An IEEE 754-1985 ISO/IEC/IEEE 60559:2011 single precision 32 bit float with 23 bits of mantissa cannot distinguish all possible 10 character values.
999999999 and 999999998 will have the same rounded value.
You would need at least 64 bit double precision to represent NUMBER(15,2) values
Senior Database Administrator
Commented:
Dug a little deeper and got around the issue by declaring the variable that's holding the value fetched from the database as BigDecimal instead of Float.

That got rid of my problems.

Thanks for the feedback, points for everyone for the assists (and the understanding of how groovy handles these things).
Steve WalesSenior Database Administrator

Author

Commented:
I solved the issue myself by using a different datatype that stores data differently, however all the experts deserve points here because of the tips I learned going through the process (novice Groovy programmer here ...)