Improve company productivity with a Business Account.Sign Up

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 273
  • Last Modified:

Math Error?

Ok, here's a fun one:

<CFOUTPUT>
<PRE>
    4.42 * 100  = #Evaluate("4.42 * 100")#<BR>
int(4.42 * 100) = #Evaluate("int(4.42 * 100)")#<BR>
    2.55 * 100  = #Evaluate("2.55 * 100")#<BR>
int(2.55 * 100) = #Evaluate("int(2.55 * 100)")#<BR>
</PRE>
</CFOUTPUT>

This outputs the following on my server:

    4.42 * 100  = 442
int(4.42 * 100) = 442
    2.55 * 100  = 255
int(2.55 * 100) = 254

This is related to the answer given in this question:
http://www.experts-exchange.com/jsp/qShow.jsp?ta=coldfusion&qid=10320322 

Since the math is wrong, it doesn't drop all the zeroes that I need it to on something like "2.550".  I don't know how frequent the error is, this is the only occurance i have seen so far.

My server is NT4.0 SP6a CF Version 4.01 - perhaps this was fixed in a later version?
0
dlewis9
Asked:
dlewis9
  • 5
  • 2
1 Solution
 
deepchandaCommented:
i would think CF function round(..) would be a more suitable in this scenario

DC
0
 
nathansCommented:
Ouch your right let me see if I can figure what is up.

hmmmm
0
 
nathansCommented:
Wow unless I am wrong I think you helped uncover a bug.

Let me check with Allaire!!
0
Get 10% Off Your First Squarespace Website

Ready to showcase your work, publish content or promote your business online? With Squarespace’s award-winning templates and 24/7 customer service, getting started is simple. Head to Squarespace.com and use offer code ‘EXPERTS’ to get 10% off your first purchase.

 
nathansCommented:
Ouch! I just heard this is so in Visual C++ as well... hmmm I want to come up with a solution... not sure why it is doing this.. but I will try to come up with a solution.
0
 
nathansCommented:

Not the solution I thought it would be but here is a solution.
======================================

<cfset x = 54.350350>
<cfset str1 = "#x#">
<cfset temp1 = RTrim(Replace(str1, "0", " ", "ALL"))>
<cfset num1 = Val(Replace(temp1, " ", "0", "ALL"))>

<cfset y = 2.55>
<cfset str2 = "#y#">
<cfset temp2 = RTrim(Replace(str2, "0", " ", "ALL"))>
<cfset num2 = Val(Replace(temp2, " ", "0", "ALL"))>

<cfoutput><br><br>
#num1#<br>
#num2#<br>
</cfoutput>
0
 
dlewis9Author Commented:
Nice creativity!  That looks like it would work, but it sure can't be very efficient :) After thinking about it again, I realized that it really doesn't matter whether int(), round(), or fix() is used, since in the process, we are just trying to chop off the decimal part and see if it makes a difference in the test value..

Strangely, the round() function returns the correct results on a test value of 2.55 -- whereas fix() had the same error as int().

I'll test out the round() function tomorrow in my application..if that doesn't work, then the string trimming method definately will..
0
 
nathansCommented:
Wow Round does work for your situation but if your trying to drop the ending zeros then surely my string solution would be best...

var x1= 23.355400
var x2= 23.354000
var x3= 23.355940
var x4= 23.355300


like example above first ends with 2 zeros, then 3 then 1 then 2 round would round up...

but if you add it to my other solution the it would work.
0
 
dlewis9Author Commented:
I hate to reject a correct answer, but since the round() function was suggested first and does seem to work properly, I'll have to go with that as "the answer", as it provides a solution to the question.

I'm starting to wonder, however, if the string solution would be more efficient than the dropzero.cfm custom tag script..guess i'll have to check that out!

Thanks nathans -- let me know if you hear back from Allaire on the possible bug (dlewis@uniden.com)..i'm curious as to why it would do that..
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Get 10% Off Your First Squarespace Website

Ready to showcase your work, publish content or promote your business online? With Squarespace’s award-winning templates and 24/7 customer service, getting started is simple. Head to Squarespace.com and use offer code ‘EXPERTS’ to get 10% off your first purchase.

  • 5
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now