Solved

Math Error?

Posted on 2000-04-04
8
264 Views
Last Modified: 2013-12-24
Ok, here's a fun one:

<CFOUTPUT>
<PRE>
    4.42 * 100  = #Evaluate("4.42 * 100")#<BR>
int(4.42 * 100) = #Evaluate("int(4.42 * 100)")#<BR>
    2.55 * 100  = #Evaluate("2.55 * 100")#<BR>
int(2.55 * 100) = #Evaluate("int(2.55 * 100)")#<BR>
</PRE>
</CFOUTPUT>

This outputs the following on my server:

    4.42 * 100  = 442
int(4.42 * 100) = 442
    2.55 * 100  = 255
int(2.55 * 100) = 254

This is related to the answer given in this question:
http://www.experts-exchange.com/jsp/qShow.jsp?ta=coldfusion&qid=10320322 

Since the math is wrong, it doesn't drop all the zeroes that I need it to on something like "2.550".  I don't know how frequent the error is, this is the only occurance i have seen so far.

My server is NT4.0 SP6a CF Version 4.01 - perhaps this was fixed in a later version?
0
Comment
Question by:dlewis9
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 2
8 Comments
 
LVL 1

Accepted Solution

by:
deepchanda earned 50 total points
ID: 2684856
i would think CF function round(..) would be a more suitable in this scenario

DC
0
 
LVL 5

Expert Comment

by:nathans
ID: 2685096
Ouch your right let me see if I can figure what is up.

hmmmm
0
 
LVL 5

Expert Comment

by:nathans
ID: 2685103
Wow unless I am wrong I think you helped uncover a bug.

Let me check with Allaire!!
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 5

Expert Comment

by:nathans
ID: 2686166
Ouch! I just heard this is so in Visual C++ as well... hmmm I want to come up with a solution... not sure why it is doing this.. but I will try to come up with a solution.
0
 
LVL 5

Expert Comment

by:nathans
ID: 2686178

Not the solution I thought it would be but here is a solution.
======================================

<cfset x = 54.350350>
<cfset str1 = "#x#">
<cfset temp1 = RTrim(Replace(str1, "0", " ", "ALL"))>
<cfset num1 = Val(Replace(temp1, " ", "0", "ALL"))>

<cfset y = 2.55>
<cfset str2 = "#y#">
<cfset temp2 = RTrim(Replace(str2, "0", " ", "ALL"))>
<cfset num2 = Val(Replace(temp2, " ", "0", "ALL"))>

<cfoutput><br><br>
#num1#<br>
#num2#<br>
</cfoutput>
0
 
LVL 2

Author Comment

by:dlewis9
ID: 2686200
Nice creativity!  That looks like it would work, but it sure can't be very efficient :) After thinking about it again, I realized that it really doesn't matter whether int(), round(), or fix() is used, since in the process, we are just trying to chop off the decimal part and see if it makes a difference in the test value..

Strangely, the round() function returns the correct results on a test value of 2.55 -- whereas fix() had the same error as int().

I'll test out the round() function tomorrow in my application..if that doesn't work, then the string trimming method definately will..
0
 
LVL 5

Expert Comment

by:nathans
ID: 2686230
Wow Round does work for your situation but if your trying to drop the ending zeros then surely my string solution would be best...

var x1= 23.355400
var x2= 23.354000
var x3= 23.355940
var x4= 23.355300


like example above first ends with 2 zeros, then 3 then 1 then 2 round would round up...

but if you add it to my other solution the it would work.
0
 
LVL 2

Author Comment

by:dlewis9
ID: 2693494
I hate to reject a correct answer, but since the round() function was suggested first and does seem to work properly, I'll have to go with that as "the answer", as it provides a solution to the question.

I'm starting to wonder, however, if the string solution would be more efficient than the dropzero.cfm custom tag script..guess i'll have to check that out!

Thanks nathans -- let me know if you hear back from Allaire on the possible bug (dlewis@uniden.com)..i'm curious as to why it would do that..
0

Featured Post

Easy, flexible multimedia distribution & control

Coming soon!  Ideal for large-scale A/V applications, ATEN's VM3200 Modular Matrix Switch is an all-in-one solution that simplifies video wall integration. Easily customize display layouts to see what you want, how you want it in 4k.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SCCM vs Windows server 5 87
Problem to go to Web page 2 131
WEB Farm 6 80
htaccess block all countries but the US 4 41
This is a guide to setting up a new WHM/cPanel Server to be used for web hosting accounts. It is intended for web hosting company administrators and dedicated server owners. For under $99 per month (considering normal rate of Big Data Cetnters like …
In our day to day coding, how many times have we come across a necessity to check whether a URL is a broken link or not? For those of you that answered countless and are using ColdFusion like myself, then this article is for you.  It will show yo…
Finding and deleting duplicate (picture) files can be a time consuming task. My wife and I, our three kids and their families all share one dilemma: Managing our pictures. Between desktops, laptops, phones, tablets, and cameras; over the last decade…

733 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question