?
Solved

Rounding problem

Posted on 2003-03-04
10
Medium Priority
?
461 Views
Last Modified: 2010-04-04
I have a program with the following code...

procedure TProfile.SetProfileLdg(Length: Single; Precision: Integer);
// Sets the Length of the Profile, according to the precision (no. of
// decimal places) given as the second parameter.
begin
     CASE Precision of
       0: Self.Length := Round(Length);
       1: Self.Length := SimpleRoundTo(Length, -1);
       2: Self.Length := SimpleRoundTo(Length, -2);
     end; {CASE}
end; {TProfile.SetProfileLdg}

After reading delphi's help on 'SimpleRoundTo', I expected a call with e.g. Profile.SetProfileLdg(100.123456, 1) to return a result of 100.1, but instead I get maybe 100.1000001234 or something similar. Where am I going wrong? Is there another, more suitable, approach to getting a real number rounded to 1 or 2 decimal places?
Any help greatly appreciated.
0
Comment
Question by:jofftee
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 2
  • 2
  • +2
10 Comments
 
LVL 3

Expert Comment

by:sfock
ID: 8067383
i'd try it with the good old
begin
    CASE Precision of
      0: Self.Length := Round(Length);
      1: Self.Length := Round(Length*10)/10;
      2: Self.Length := Round(Length*100)/100;
    end; {CASE}
end; {TProfile.SetProfileLdg}
0
 
LVL 3

Expert Comment

by:ILE
ID: 8067450
wery simle round(

 this function roud to the nearest 0.05

function round05(a:real):real;
begin
round05:=round(a*20)/20;
end;

this function round to 2 decimals

function round05(a:real):real;
begin
round05:=round(a*100)/100;
end;


this function round to 4 decimals

function round05(a:real):real;
begin
round05:=round(a*10000)/10000;
end;


you get the idea :)))


thanks
0
 
LVL 3

Expert Comment

by:sfock
ID: 8067537
or probably a little bit more generic like this:

var
  decs : integer;
  floater : double;
begin
  floater := IntPower(10, Precision);
  if floater = 0 then
    floater := 1;
  Self.Length := round(Length * floater) / floater;
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 6

Expert Comment

by:swift99
ID: 8069599
The reason you are getting this wierd value is because the result you are receiving represent does not have an exact representation in binary to the precision of the floating point entity returned by the function.

Unlike COBOL, which stores its numbers in BCD (Binary Coded Decimal) format, Delphi stores numbers in true binary format (base two).  

In COBOL, a PIC 999.9 would store the number 100.1 as the digits 100.1, and compute in base 10 at the expense of some CPU.  

In Object Pascal (Delphi), the number 100.1 is stored as a double precision binary floating point number, so 100.1 is represented by a whole number (1), a fraction (1/1000), and an exponent (2), which is 1.001 times 10 to the power of 2.  1/1000 represents nicely in base 10, but in base two it is quite nasty.

Delphi does support BCD fixed point Currency and COMP types specifically for financial applications where these rounding issues are critical.

When you convert it to base 10 digits for display, the limits to the precision of the floating point representation become apparent.  

We have the same problem in base 10 when representing fractions of multiples of three.  When you express 1/3 to five places, you see 0.33333.  However, 0.33333 is not equal to 1/3.
0
 
LVL 17

Expert Comment

by:geobul
ID: 8070267
Hi,

This is a matter of displaying/using real numbers. For displaying/writing to a file use FloatToStrF or Format functions.

var
  StringVar: string;
  FloatVar: double;
begin
  FloatVar := 100.10001234;
  StringVar := Format('%10.2f',[FloatVar]);
  ShowMessage(StringVar);
end;

Regards, Geo
0
 

Author Comment

by:jofftee
ID: 8070921
Thanks for the comments received.
I have tried the code
CASE Precision of
     0: Self.Length := Round(Length);
     1: Self.Length := Round(Length*10)/10;
     2: Self.Length := Round(Length*100)/100;
end; {CASE}
as suggested but it doesn't work any better than my original code.
After altering the TProfile.Length field from a single to a currency type, I can set the 'CurrencyDecimals' variable, and in association with the above, achieve the desired rounding. This could of course be achieved with the FloatToStrF function, as suggested in the last comment, but I would rather have the value stored correctly formatted, rather than outputted correctly formatted.
Will using a currency type instead of a single create problems?
0
 
LVL 3

Accepted Solution

by:
sfock earned 225 total points
ID: 8071390
after i couldn't reproduce your issue at first i now found out that your problem is not a rounding problem.

The problem is that you use 4 byte single values. If you use Round like i suggestet it returns a Int64 if you then divde it by 10 the resulting type will be a read wich (if using default compilersettings) is a 8 Bytes long real --> a double.

when you assign a double to single value 4 bytes will be lost. After these types are floating point types storing the data in mantissa it does not result simply loosing some digits.

You might veryfiy the following:

procedure test;
var
  myDouble : double;
  mySingle : single;
begin
  myDouble := 100.12;
  mySingle := myDouble;
  showMessage(FloatToStr(myDouble));
end;

The message box will not show 100.12 as expectet
0
 
LVL 3

Expert Comment

by:sfock
ID: 8071439
oh well and forgot to say : currency should not have this problems.
Anyhow i'd suggest to use double (if it isn't really a currency value) and rond like suggestet
0
 
LVL 6

Expert Comment

by:swift99
ID: 8072021
Currency is actually stored as BCD, so for numbers within the range and precision of the variable it should be fine.  Not knowing the mathematical transforms you are applying, I can't predict in advance the exact problems you might run into.

0
 

Author Comment

by:jofftee
ID: 8072237
Works fine as a double.
Thanks for the help.
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
NetCrunch network monitor is a highly extensive platform for network monitoring and alert generation. In this video you'll see a live demo of NetCrunch with most notable features explained in a walk-through manner. You'll also get to know the philos…
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…
Suggested Courses
Course of the Month9 days, 12 hours left to enroll

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question