Solved

XPath and XQuery, formating and precision

Posted on 2015-01-22
3
83 Views
Last Modified: 2015-01-22
From my fn:sum command, I get the following:

1.0001865499999999E6

Open in new window


where I should be getting

1000186.55

Open in new window


How do I fix that
0
Comment
Question by:Anthony Lucia
  • 2
3 Comments
 

Author Comment

by:Anthony Lucia
ID: 40565336
I do understand that perhaps a format-number() function might help, but what would cause the lack of precision
0
 
LVL 60

Accepted Solution

by:
Geert Bormans earned 500 total points
ID: 40565362
format-number() would fix that, but not entirely

numbers in XPath are of type xs:double
xs:double has floating point... so they are not decimals
avoid the issue by explicitely casting to xs:decimal prior to the sum()

instead of sum(node) do sum(node/xs:decimal())

if you are bound to XPath 1.0 it might be a lot trickier
0
 
LVL 60

Assisted Solution

by:Geert Bormans
Geert Bormans earned 500 total points
ID: 40565370
http://en.wikipedia.org/wiki/Floating_point
has a section on accuracy issues
decimals don't have that
0

Featured Post

Space-Age Communications Transitions to DevOps

ViaSat, a global provider of satellite and wireless communications, securely connects businesses, governments, and organizations to the Internet. Learn how ViaSat’s Network Solutions Engineer, drove the transition from a traditional network support to a DevOps-centric model.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
jboss wildfly 10.1 10 228
XSLT Display Label Name 1 27
PowerShell script to remove string in xml file 8 32
Java Eclipse Loop 3 20
Introduction In my previous article (http://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/SSIS/A_9150-Loading-XML-Using-SSIS.html) I showed you how the XML Source component can be used to load XML files into a SQL Server database, us…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers will learn about if statements in Java and their use The if statement: The condition required to create an if statement: Variations of if statements: An example using if statements:
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question