Solved

XPath and XQuery, formating and precision

Posted on 2015-01-22
3
88 Views
Last Modified: 2015-01-22
From my fn:sum command, I get the following:

1.0001865499999999E6

Open in new window


where I should be getting

1000186.55

Open in new window


How do I fix that
0
Comment
Question by:Anthony Lucia
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 

Author Comment

by:Anthony Lucia
ID: 40565336
I do understand that perhaps a format-number() function might help, but what would cause the lack of precision
0
 
LVL 60

Accepted Solution

by:
Geert Bormans earned 500 total points
ID: 40565362
format-number() would fix that, but not entirely

numbers in XPath are of type xs:double
xs:double has floating point... so they are not decimals
avoid the issue by explicitely casting to xs:decimal prior to the sum()

instead of sum(node) do sum(node/xs:decimal())

if you are bound to XPath 1.0 it might be a lot trickier
0
 
LVL 60

Assisted Solution

by:Geert Bormans
Geert Bormans earned 500 total points
ID: 40565370
http://en.wikipedia.org/wiki/Floating_point
has a section on accuracy issues
decimals don't have that
0

Featured Post

Creating Instructional Tutorials  

For Any Use & On Any Platform

Contextual Guidance at the moment of need helps your employees/users adopt software o& achieve even the most complex tasks instantly. Boost knowledge retention, software adoption & employee engagement with easy solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For beginner Java programmers or at least those new to the Eclipse IDE, the following tutorial will show some (four) ways in which you can import your Java projects to your Eclipse workbench. Introduction While learning Java can be done with…
In this post we will learn different types of Android Layout and some basics of an Android App.
The viewer will learn how to implement Singleton Design Pattern in Java.
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.

615 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question