• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 472
  • Last Modified:

Rounding problem

I have a program with the following code...

procedure TProfile.SetProfileLdg(Length: Single; Precision: Integer);
// Sets the Length of the Profile, according to the precision (no. of
// decimal places) given as the second parameter.
begin
     CASE Precision of
       0: Self.Length := Round(Length);
       1: Self.Length := SimpleRoundTo(Length, -1);
       2: Self.Length := SimpleRoundTo(Length, -2);
     end; {CASE}
end; {TProfile.SetProfileLdg}

After reading delphi's help on 'SimpleRoundTo', I expected a call with e.g. Profile.SetProfileLdg(100.123456, 1) to return a result of 100.1, but instead I get maybe 100.1000001234 or something similar. Where am I going wrong? Is there another, more suitable, approach to getting a real number rounded to 1 or 2 decimal places?
Any help greatly appreciated.
0
jofftee
Asked:
jofftee
  • 4
  • 2
  • 2
  • +2
1 Solution
 
sfockCommented:
i'd try it with the good old
begin
    CASE Precision of
      0: Self.Length := Round(Length);
      1: Self.Length := Round(Length*10)/10;
      2: Self.Length := Round(Length*100)/100;
    end; {CASE}
end; {TProfile.SetProfileLdg}
0
 
ILECommented:
wery simle round(

 this function roud to the nearest 0.05

function round05(a:real):real;
begin
round05:=round(a*20)/20;
end;

this function round to 2 decimals

function round05(a:real):real;
begin
round05:=round(a*100)/100;
end;


this function round to 4 decimals

function round05(a:real):real;
begin
round05:=round(a*10000)/10000;
end;


you get the idea :)))


thanks
0
 
sfockCommented:
or probably a little bit more generic like this:

var
  decs : integer;
  floater : double;
begin
  floater := IntPower(10, Precision);
  if floater = 0 then
    floater := 1;
  Self.Length := round(Length * floater) / floater;
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
swift99Commented:
The reason you are getting this wierd value is because the result you are receiving represent does not have an exact representation in binary to the precision of the floating point entity returned by the function.

Unlike COBOL, which stores its numbers in BCD (Binary Coded Decimal) format, Delphi stores numbers in true binary format (base two).  

In COBOL, a PIC 999.9 would store the number 100.1 as the digits 100.1, and compute in base 10 at the expense of some CPU.  

In Object Pascal (Delphi), the number 100.1 is stored as a double precision binary floating point number, so 100.1 is represented by a whole number (1), a fraction (1/1000), and an exponent (2), which is 1.001 times 10 to the power of 2.  1/1000 represents nicely in base 10, but in base two it is quite nasty.

Delphi does support BCD fixed point Currency and COMP types specifically for financial applications where these rounding issues are critical.

When you convert it to base 10 digits for display, the limits to the precision of the floating point representation become apparent.  

We have the same problem in base 10 when representing fractions of multiples of three.  When you express 1/3 to five places, you see 0.33333.  However, 0.33333 is not equal to 1/3.
0
 
geobulCommented:
Hi,

This is a matter of displaying/using real numbers. For displaying/writing to a file use FloatToStrF or Format functions.

var
  StringVar: string;
  FloatVar: double;
begin
  FloatVar := 100.10001234;
  StringVar := Format('%10.2f',[FloatVar]);
  ShowMessage(StringVar);
end;

Regards, Geo
0
 
joffteeAuthor Commented:
Thanks for the comments received.
I have tried the code
CASE Precision of
     0: Self.Length := Round(Length);
     1: Self.Length := Round(Length*10)/10;
     2: Self.Length := Round(Length*100)/100;
end; {CASE}
as suggested but it doesn't work any better than my original code.
After altering the TProfile.Length field from a single to a currency type, I can set the 'CurrencyDecimals' variable, and in association with the above, achieve the desired rounding. This could of course be achieved with the FloatToStrF function, as suggested in the last comment, but I would rather have the value stored correctly formatted, rather than outputted correctly formatted.
Will using a currency type instead of a single create problems?
0
 
sfockCommented:
after i couldn't reproduce your issue at first i now found out that your problem is not a rounding problem.

The problem is that you use 4 byte single values. If you use Round like i suggestet it returns a Int64 if you then divde it by 10 the resulting type will be a read wich (if using default compilersettings) is a 8 Bytes long real --> a double.

when you assign a double to single value 4 bytes will be lost. After these types are floating point types storing the data in mantissa it does not result simply loosing some digits.

You might veryfiy the following:

procedure test;
var
  myDouble : double;
  mySingle : single;
begin
  myDouble := 100.12;
  mySingle := myDouble;
  showMessage(FloatToStr(myDouble));
end;

The message box will not show 100.12 as expectet
0
 
sfockCommented:
oh well and forgot to say : currency should not have this problems.
Anyhow i'd suggest to use double (if it isn't really a currency value) and rond like suggestet
0
 
swift99Commented:
Currency is actually stored as BCD, so for numbers within the range and precision of the variable it should be fine.  Not knowing the mathematical transforms you are applying, I can't predict in advance the exact problems you might run into.

0
 
joffteeAuthor Commented:
Works fine as a double.
Thanks for the help.
0

Featured Post

Become an Android App Developer

Ready to kick start your career in 2018? Learn how to build an Android app in January’s Course of the Month and open the door to new opportunities.

  • 4
  • 2
  • 2
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now