Solved

NumberVar Rounding/Integer Problem

Posted on 2010-11-23
10
1,838 Views
Last Modified: 2012-05-10
We have an issue with a report that is returning the incorrect value.

We use the following formulas:

CALCVAR (Report Header1)
Whileprintingrecords;
Numbervar justabove;
justabove:= 0.1;

CALCANALYSIS (Report Header2)
Whileprintingrecords;
Numbervar justabove;
Stringvar analysis;
If {field1} – {field2} >= just above then analysis:= “just below average”
Note: (The mathematical calculation of the two fields is 0.1)  

DISPLAYANALYSIS (Report Footer 1)
Whileprintingrecords;
Stringvar analysis;

When DISPLAYANALYSIS is used in the report it returns nothing.  
Is this something to do with rounding? Or using numbvar to store 0.1 as it works with other numbers such as 1.5 (if I change justabove:= 0.1 to justabove:= 0.5 and update the database fields to match)

Please help, I am sure I must be doing something daft.
0
Comment
Question by:optimumreports
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 77

Accepted Solution

by:
peter57r earned 200 total points
ID: 34196047
It's not obvious to me that it's a precision problem although i won't rule it out because accuracy of decimals is always a potential issue.


Perhaps it would be worth testing a variation of this. Set justabove to 1 and do..

If int({field1}*10) – int({field2}*10) >= justabove then analysis:= “just below average”
0
 
LVL 101

Assisted Solution

by:mlmcc
mlmcc earned 150 total points
ID: 34197202
I suspect it is a rounding issue.  remember computers use base 2 or binary.  This limits the values that can be displayed/saved as exact values.

0.1 cannot be saved exactly so I think you are getting a round problem

One way to check would be as peter suggests.

mlmcc
0
 
LVL 35

Assisted Solution

by:James0628
James0628 earned 150 total points
ID: 34203400
FWIW, I did a couple of quick tests and your formulas seem to work fine here.  i wouldn't really expect precision to be a problem, unless your values are unusually large, or have a lot of decimal places.

 I'd try changing the last line in the second formula to just

{field1} - {field2}

 and see what you get.

 James
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 1

Author Comment

by:optimumreports
ID: 34204465
Thanks guys...

We'll test this this afternoon and feedback to you all.

Cheers,

Tom
0
 
LVL 1

Author Comment

by:optimumreports
ID: 34231190
Many thanks for the suggestions sorry for my slow delay in getting back to you.

I have tried the following as suggested but it still returns blank data:

1

whileprintingrecords;
Numbervar justabove;

justabove:= 1; // Set from 0.1 to 1 as suggested

2

whileprintingrecords;
numbervar justabove;

stringvar holding_Writing;

if {KS1_All.Writing Significant} = true and {KS1_All.Writing APS} > {National_All.Writing APS} then holding_Writing:= "significantly above"
else if {KS1_All.Writing Significant} = true and {KS1_All.Writing APS} < {National_All.Writing APS} then holding_Writing:= "significantly below"
else

(
if  int({KS1_All.Writing APS}*10) - int({National_All.Writing APS}*10)  >= justabove then holding_Writing:= "just above"
);

Any ideas?

Tom
0
 
LVL 101

Expert Comment

by:mlmcc
ID: 34231706
Does the report show the values of the fields being used?
If not can you put them on the report and show us what you are getting

mlmcc
0
 
LVL 35

Expert Comment

by:James0628
ID: 34236617
Or, as I suggested before, just change the last line in your second formula to {field1} - {field2}.  The point of that is to see the actual value that CR is getting from that calculation.  Of course the assumption there is that the result of that formula is visible on the report (not in a suppressed field or section).  And you may need/want to change the field format to show more decimal places, to get a more precise view of the value.

 James
0
 
LVL 1

Author Comment

by:optimumreports
ID: 34248822
Hi all,

When we use the following integer formula it returns the data correctly which is great:

if  int({KS1_All.Writing APS}*10) - int({National_All.Writing APS}*10)  >= justabove then holding_Writing:= "just above"

However, now the integer formula is working when mutiplying by 10, I would just like to to confirm if the precision problem identified would affect any of the following variables or is it just where it is 0.1 and -0.1?

verysigabove:=3
sigabove:=2
wellabove:=1
above:=0.5
justabove:=1 \\previuously 0.1
verysigbelow:=-3
sigbelow:=-2
wellbelow:=-1
below:=-0.5
justbelow:=-1 \\previuously -0.1
inline:=0

Or will I need to mutiply all instances by 10?

Cheers,

Tom
0
 
LVL 101

Expert Comment

by:mlmcc
ID: 34249150
Since integers can be represent exactly it shouldn't affect any of them.  Even .5 is represented exactly so that should be ok

mlmcc
0
 
LVL 1

Author Closing Comment

by:optimumreports
ID: 34284013
Thanks for the feedback. Sorry we didnt get straight back to you all - we are really busy here at the moment.

Thanks again for your help - much appreciated.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hot fix for .Net Crystal Reports 10.2.3600.0 to fix problems with sub reports running on 64 bit operating systems ISSUE: Reports which contain subreports fail with error "Missing Parameter Value" DEPLOYMENT SERVER OS: Windows 2008 with 64 bi…
There have always been a lot of questions related to when Crystal Reports evaluates report components (such as formulas, summaries, cross-tabs, charts, to name a few examples). Crystal Reports uses a two-pass reporting process to provide greater …
In this video, viewers are given an introduction to using the Windows 10 Snipping Tool, how to quickly locate it when it's needed and also how make it always available with a single click of a mouse button, by pinning it to the Desktop Task Bar. Int…
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…

622 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question