optimumreports
asked on
NumberVar Rounding/Integer Problem
We have an issue with a report that is returning the incorrect value.
We use the following formulas:
CALCVAR (Report Header1)
Whileprintingrecords;
Numbervar justabove;
justabove:= 0.1;
CALCANALYSIS (Report Header2)
Whileprintingrecords;
Numbervar justabove;
Stringvar analysis;
If {field1} – {field2} >= just above then analysis:= “just below average”
Note: (The mathematical calculation of the two fields is 0.1)
DISPLAYANALYSIS (Report Footer 1)
Whileprintingrecords;
Stringvar analysis;
When DISPLAYANALYSIS is used in the report it returns nothing.
Is this something to do with rounding? Or using numbvar to store 0.1 as it works with other numbers such as 1.5 (if I change justabove:= 0.1 to justabove:= 0.5 and update the database fields to match)
Please help, I am sure I must be doing something daft.
We use the following formulas:
CALCVAR (Report Header1)
Whileprintingrecords;
Numbervar justabove;
justabove:= 0.1;
CALCANALYSIS (Report Header2)
Whileprintingrecords;
Numbervar justabove;
Stringvar analysis;
If {field1} – {field2} >= just above then analysis:= “just below average”
Note: (The mathematical calculation of the two fields is 0.1)
DISPLAYANALYSIS (Report Footer 1)
Whileprintingrecords;
Stringvar analysis;
When DISPLAYANALYSIS is used in the report it returns nothing.
Is this something to do with rounding? Or using numbvar to store 0.1 as it works with other numbers such as 1.5 (if I change justabove:= 0.1 to justabove:= 0.5 and update the database fields to match)
Please help, I am sure I must be doing something daft.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Many thanks for the suggestions sorry for my slow delay in getting back to you.
I have tried the following as suggested but it still returns blank data:
1
whileprintingrecords;
Numbervar justabove;
justabove:= 1; // Set from 0.1 to 1 as suggested
2
whileprintingrecords;
numbervar justabove;
stringvar holding_Writing;
if {KS1_All.Writing Significant} = true and {KS1_All.Writing APS} > {National_All.Writing APS} then holding_Writing:= "significantly above"
else if {KS1_All.Writing Significant} = true and {KS1_All.Writing APS} < {National_All.Writing APS} then holding_Writing:= "significantly below"
else
(
if int({KS1_All.Writing APS}*10) - int({National_All.Writing APS}*10) >= justabove then holding_Writing:= "just above"
);
Any ideas?
Tom
I have tried the following as suggested but it still returns blank data:
1
whileprintingrecords;
Numbervar justabove;
justabove:= 1; // Set from 0.1 to 1 as suggested
2
whileprintingrecords;
numbervar justabove;
stringvar holding_Writing;
if {KS1_All.Writing Significant} = true and {KS1_All.Writing APS} > {National_All.Writing APS} then holding_Writing:= "significantly above"
else if {KS1_All.Writing Significant} = true and {KS1_All.Writing APS} < {National_All.Writing APS} then holding_Writing:= "significantly below"
else
(
if int({KS1_All.Writing APS}*10) - int({National_All.Writing APS}*10) >= justabove then holding_Writing:= "just above"
);
Any ideas?
Tom
Does the report show the values of the fields being used?
If not can you put them on the report and show us what you are getting
mlmcc
If not can you put them on the report and show us what you are getting
mlmcc
Or, as I suggested before, just change the last line in your second formula to {field1} - {field2}. The point of that is to see the actual value that CR is getting from that calculation. Of course the assumption there is that the result of that formula is visible on the report (not in a suppressed field or section). And you may need/want to change the field format to show more decimal places, to get a more precise view of the value.
James
James
ASKER
Hi all,
When we use the following integer formula it returns the data correctly which is great:
if int({KS1_All.Writing APS}*10) - int({National_All.Writing APS}*10) >= justabove then holding_Writing:= "just above"
However, now the integer formula is working when mutiplying by 10, I would just like to to confirm if the precision problem identified would affect any of the following variables or is it just where it is 0.1 and -0.1?
verysigabove:=3
sigabove:=2
wellabove:=1
above:=0.5
justabove:=1 \\previuously 0.1
verysigbelow:=-3
sigbelow:=-2
wellbelow:=-1
below:=-0.5
justbelow:=-1 \\previuously -0.1
inline:=0
Or will I need to mutiply all instances by 10?
Cheers,
Tom
When we use the following integer formula it returns the data correctly which is great:
if int({KS1_All.Writing APS}*10) - int({National_All.Writing APS}*10) >= justabove then holding_Writing:= "just above"
However, now the integer formula is working when mutiplying by 10, I would just like to to confirm if the precision problem identified would affect any of the following variables or is it just where it is 0.1 and -0.1?
verysigabove:=3
sigabove:=2
wellabove:=1
above:=0.5
justabove:=1 \\previuously 0.1
verysigbelow:=-3
sigbelow:=-2
wellbelow:=-1
below:=-0.5
justbelow:=-1 \\previuously -0.1
inline:=0
Or will I need to mutiply all instances by 10?
Cheers,
Tom
Since integers can be represent exactly it shouldn't affect any of them. Even .5 is represented exactly so that should be ok
mlmcc
mlmcc
ASKER
Thanks for the feedback. Sorry we didnt get straight back to you all - we are really busy here at the moment.
Thanks again for your help - much appreciated.
Thanks again for your help - much appreciated.
ASKER
We'll test this this afternoon and feedback to you all.
Cheers,
Tom