Solved

XPath and XQuery, formating and precision

Posted on 2015-01-22
3
86 Views
Last Modified: 2015-01-22
From my fn:sum command, I get the following:

1.0001865499999999E6

Open in new window


where I should be getting

1000186.55

Open in new window


How do I fix that
0
Comment
Question by:Anthony Lucia
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 

Author Comment

by:Anthony Lucia
ID: 40565336
I do understand that perhaps a format-number() function might help, but what would cause the lack of precision
0
 
LVL 60

Accepted Solution

by:
Geert Bormans earned 500 total points
ID: 40565362
format-number() would fix that, but not entirely

numbers in XPath are of type xs:double
xs:double has floating point... so they are not decimals
avoid the issue by explicitely casting to xs:decimal prior to the sum()

instead of sum(node) do sum(node/xs:decimal())

if you are bound to XPath 1.0 it might be a lot trickier
0
 
LVL 60

Assisted Solution

by:Geert Bormans
Geert Bormans earned 500 total points
ID: 40565370
http://en.wikipedia.org/wiki/Floating_point
has a section on accuracy issues
decimals don't have that
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Viewers learn about the third conditional statement “else if” and use it in an example program. Then additional information about conditional statements is provided, covering the topic thoroughly. Viewers learn about the third conditional statement …
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question