Solved

Math Error?

Posted on 2000-04-04
8
262 Views
Last Modified: 2013-12-24
Ok, here's a fun one:

<CFOUTPUT>
<PRE>
    4.42 * 100  = #Evaluate("4.42 * 100")#<BR>
int(4.42 * 100) = #Evaluate("int(4.42 * 100)")#<BR>
    2.55 * 100  = #Evaluate("2.55 * 100")#<BR>
int(2.55 * 100) = #Evaluate("int(2.55 * 100)")#<BR>
</PRE>
</CFOUTPUT>

This outputs the following on my server:

    4.42 * 100  = 442
int(4.42 * 100) = 442
    2.55 * 100  = 255
int(2.55 * 100) = 254

This is related to the answer given in this question:
http://www.experts-exchange.com/jsp/qShow.jsp?ta=coldfusion&qid=10320322 

Since the math is wrong, it doesn't drop all the zeroes that I need it to on something like "2.550".  I don't know how frequent the error is, this is the only occurance i have seen so far.

My server is NT4.0 SP6a CF Version 4.01 - perhaps this was fixed in a later version?
0
Comment
Question by:dlewis9
  • 5
  • 2
8 Comments
 
LVL 1

Accepted Solution

by:
deepchanda earned 50 total points
ID: 2684856
i would think CF function round(..) would be a more suitable in this scenario

DC
0
 
LVL 5

Expert Comment

by:nathans
ID: 2685096
Ouch your right let me see if I can figure what is up.

hmmmm
0
 
LVL 5

Expert Comment

by:nathans
ID: 2685103
Wow unless I am wrong I think you helped uncover a bug.

Let me check with Allaire!!
0
Use Case: Protecting a Hybrid Cloud Infrastructure

Microsoft Azure is rapidly becoming the norm in dynamic IT environments. This document describes the challenges that organizations face when protecting data in a hybrid cloud IT environment and presents a use case to demonstrate how Acronis Backup protects all data.

 
LVL 5

Expert Comment

by:nathans
ID: 2686166
Ouch! I just heard this is so in Visual C++ as well... hmmm I want to come up with a solution... not sure why it is doing this.. but I will try to come up with a solution.
0
 
LVL 5

Expert Comment

by:nathans
ID: 2686178

Not the solution I thought it would be but here is a solution.
======================================

<cfset x = 54.350350>
<cfset str1 = "#x#">
<cfset temp1 = RTrim(Replace(str1, "0", " ", "ALL"))>
<cfset num1 = Val(Replace(temp1, " ", "0", "ALL"))>

<cfset y = 2.55>
<cfset str2 = "#y#">
<cfset temp2 = RTrim(Replace(str2, "0", " ", "ALL"))>
<cfset num2 = Val(Replace(temp2, " ", "0", "ALL"))>

<cfoutput><br><br>
#num1#<br>
#num2#<br>
</cfoutput>
0
 
LVL 2

Author Comment

by:dlewis9
ID: 2686200
Nice creativity!  That looks like it would work, but it sure can't be very efficient :) After thinking about it again, I realized that it really doesn't matter whether int(), round(), or fix() is used, since in the process, we are just trying to chop off the decimal part and see if it makes a difference in the test value..

Strangely, the round() function returns the correct results on a test value of 2.55 -- whereas fix() had the same error as int().

I'll test out the round() function tomorrow in my application..if that doesn't work, then the string trimming method definately will..
0
 
LVL 5

Expert Comment

by:nathans
ID: 2686230
Wow Round does work for your situation but if your trying to drop the ending zeros then surely my string solution would be best...

var x1= 23.355400
var x2= 23.354000
var x3= 23.355940
var x4= 23.355300


like example above first ends with 2 zeros, then 3 then 1 then 2 round would round up...

but if you add it to my other solution the it would work.
0
 
LVL 2

Author Comment

by:dlewis9
ID: 2693494
I hate to reject a correct answer, but since the round() function was suggested first and does seem to work properly, I'll have to go with that as "the answer", as it provides a solution to the question.

I'm starting to wonder, however, if the string solution would be more efficient than the dropzero.cfm custom tag script..guess i'll have to check that out!

Thanks nathans -- let me know if you hear back from Allaire on the possible bug (dlewis@uniden.com)..i'm curious as to why it would do that..
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In our day to day coding, how many times have we come across a necessity to check whether a URL is a broken link or not? For those of you that answered countless and are using ColdFusion like myself, then this article is for you.  It will show yo…
Periodically we have to update or add SSL certificates for customers. Depending upon your hosting plan you may be responsible for the installation and/or key generation. In the wake of Heartbleed many sites were forced to re-key. We will concen…
This Micro Tutorial will give you a basic overview how to record your screen with Microsoft Expression Encoder. This program is still free and open for the public to download. This will be demonstrated using Microsoft Expression Encoder 4.
Although Jacob Bernoulli (1654-1705) has been credited as the creator of "Binomial Distribution Table", Gottfried Leibniz (1646-1716) did his dissertation on the subject in 1666; Leibniz you may recall is the co-inventor of "Calculus" and beat Isaac…

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question