?
Solved

Math Error?

Posted on 2000-04-04
8
Medium Priority
?
268 Views
Last Modified: 2013-12-24
Ok, here's a fun one:

<CFOUTPUT>
<PRE>
    4.42 * 100  = #Evaluate("4.42 * 100")#<BR>
int(4.42 * 100) = #Evaluate("int(4.42 * 100)")#<BR>
    2.55 * 100  = #Evaluate("2.55 * 100")#<BR>
int(2.55 * 100) = #Evaluate("int(2.55 * 100)")#<BR>
</PRE>
</CFOUTPUT>

This outputs the following on my server:

    4.42 * 100  = 442
int(4.42 * 100) = 442
    2.55 * 100  = 255
int(2.55 * 100) = 254

This is related to the answer given in this question:
http://www.experts-exchange.com/jsp/qShow.jsp?ta=coldfusion&qid=10320322 

Since the math is wrong, it doesn't drop all the zeroes that I need it to on something like "2.550".  I don't know how frequent the error is, this is the only occurance i have seen so far.

My server is NT4.0 SP6a CF Version 4.01 - perhaps this was fixed in a later version?
0
Comment
Question by:dlewis9
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 2
8 Comments
 
LVL 1

Accepted Solution

by:
deepchanda earned 150 total points
ID: 2684856
i would think CF function round(..) would be a more suitable in this scenario

DC
0
 
LVL 5

Expert Comment

by:nathans
ID: 2685096
Ouch your right let me see if I can figure what is up.

hmmmm
0
 
LVL 5

Expert Comment

by:nathans
ID: 2685103
Wow unless I am wrong I think you helped uncover a bug.

Let me check with Allaire!!
0
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

 
LVL 5

Expert Comment

by:nathans
ID: 2686166
Ouch! I just heard this is so in Visual C++ as well... hmmm I want to come up with a solution... not sure why it is doing this.. but I will try to come up with a solution.
0
 
LVL 5

Expert Comment

by:nathans
ID: 2686178

Not the solution I thought it would be but here is a solution.
======================================

<cfset x = 54.350350>
<cfset str1 = "#x#">
<cfset temp1 = RTrim(Replace(str1, "0", " ", "ALL"))>
<cfset num1 = Val(Replace(temp1, " ", "0", "ALL"))>

<cfset y = 2.55>
<cfset str2 = "#y#">
<cfset temp2 = RTrim(Replace(str2, "0", " ", "ALL"))>
<cfset num2 = Val(Replace(temp2, " ", "0", "ALL"))>

<cfoutput><br><br>
#num1#<br>
#num2#<br>
</cfoutput>
0
 
LVL 2

Author Comment

by:dlewis9
ID: 2686200
Nice creativity!  That looks like it would work, but it sure can't be very efficient :) After thinking about it again, I realized that it really doesn't matter whether int(), round(), or fix() is used, since in the process, we are just trying to chop off the decimal part and see if it makes a difference in the test value..

Strangely, the round() function returns the correct results on a test value of 2.55 -- whereas fix() had the same error as int().

I'll test out the round() function tomorrow in my application..if that doesn't work, then the string trimming method definately will..
0
 
LVL 5

Expert Comment

by:nathans
ID: 2686230
Wow Round does work for your situation but if your trying to drop the ending zeros then surely my string solution would be best...

var x1= 23.355400
var x2= 23.354000
var x3= 23.355940
var x4= 23.355300


like example above first ends with 2 zeros, then 3 then 1 then 2 round would round up...

but if you add it to my other solution the it would work.
0
 
LVL 2

Author Comment

by:dlewis9
ID: 2693494
I hate to reject a correct answer, but since the round() function was suggested first and does seem to work properly, I'll have to go with that as "the answer", as it provides a solution to the question.

I'm starting to wonder, however, if the string solution would be more efficient than the dropzero.cfm custom tag script..guess i'll have to check that out!

Thanks nathans -- let me know if you hear back from Allaire on the possible bug (dlewis@uniden.com)..i'm curious as to why it would do that..
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you don't have the right permissions set for your WordPress location in IIS, you won't be able to perform automatic updates. Here's how to fix the problem.
Meet the world's only “Transparent Cloud™” from Superb Internet Corporation. Now, you can experience firsthand a cloud platform that consistently outperforms Amazon Web Services (AWS), IBM’s Softlayer, and Microsoft’s Azure when it comes to CPU and …
In this video you will find out how to export Office 365 mailboxes using the built in eDiscovery tool. Bear in mind that although this method might be useful in some cases, using PST files as Office 365 backup is troublesome in a long run (more on t…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question