Solved

XPath and XQuery, formating and precision

Posted on 2015-01-22
3
82 Views
Last Modified: 2015-01-22
From my fn:sum command, I get the following:

1.0001865499999999E6

Open in new window


where I should be getting

1000186.55

Open in new window


How do I fix that
0
Comment
Question by:Anthony Lucia
  • 2
3 Comments
 

Author Comment

by:Anthony Lucia
ID: 40565336
I do understand that perhaps a format-number() function might help, but what would cause the lack of precision
0
 
LVL 60

Accepted Solution

by:
Geert Bormans earned 500 total points
ID: 40565362
format-number() would fix that, but not entirely

numbers in XPath are of type xs:double
xs:double has floating point... so they are not decimals
avoid the issue by explicitely casting to xs:decimal prior to the sum()

instead of sum(node) do sum(node/xs:decimal())

if you are bound to XPath 1.0 it might be a lot trickier
0
 
LVL 60

Assisted Solution

by:Geert Bormans
Geert Bormans earned 500 total points
ID: 40565370
http://en.wikipedia.org/wiki/Floating_point
has a section on accuracy issues
decimals don't have that
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Issue with XSLT mapping 10 35
How do I remove an object from a 3 23
CSV file parsing thru Java 13 26
printf performancy 11 33
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Many times as a report developer I've been asked to display normalized data such as three rows with values Jack, Joe, and Bob as a single comma-separated string such as 'Jack, Joe, Bob', and vice versa.  Here's how to do it. 
Viewers learn about the “while” loop and how to utilize it correctly in Java. Additionally, viewers begin exploring how to include conditional statements within a while loop and avoid an endless loop. Define While Loop: Basic Example: Explanatio…
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.

786 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question