Solved

Why do these yield different results ?

Posted on 2009-05-18
11
383 Views
Last Modified: 2012-06-21
As you can see by the code the latter query has more decimals why cant I get the first query to yield the same result ?
SELECT A,B,A/B FROM
(
     SELECT colA, colB FROM thetable
) C
		
SELECT A,B,A/B FROM
(				
     SELECT		1.00000000000000000000 AS A, 1.40673489600528227000 AS B
) C
 
And the Result
 
1.00000000000000000000	1.40673489600528227000	0.710865
1.00000000000000000000	1.40673489600528227000	0.71086599389814597

Open in new window

0
Comment
Question by:Pall Palsson
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1
11 Comments
 
LVL 29

Expert Comment

by:QPR
ID: 24418247
my *guess* is that it has something to do with the datatypes in thetable.
I can't explain why from your results but:
select 1.00000000000000000000/convert(float,1.40673489600528227000) = 0.710865993898146
and
select 1.00000000000000000000/convert(real,1.40673489600528227000) = 0.710866
0
 
LVL 31

Expert Comment

by:RiteshShah
ID: 24418349
it is completely depends on the datatype you use in your table or in your calculation. have a look at following queries and its results printed after query.



SELECT A,B,A/B FROM
(                               
     SELECT             1.00000000000000000000 AS A, 1.40673489600528227000 AS B
) C --0.71086599389814597
 
 
SELECT A,B,A/B FROM
(                               
     SELECT             1.00000000000000000000 AS A, 1.40 AS B
) C --0.714285714285714285714285
 
SELECT A,B,A/B FROM
(                               
     SELECT             1 AS A, 1.40 AS B
) C --0.714285

Open in new window

0
 
LVL 4

Expert Comment

by:TimSledge
ID: 24418494
Use the format below to make sure division occurs properly and to control decimal output of the result:
SELECT convert(decimal(18,2), (convert(decimal(18,2), A)  /  convert(decimal(18,2), D) ) )

See http://www.experts-exchange.com/Programming/Languages/SQL_Syntax/Q_24416319.html
0
Increase Agility with Enabled Toolchains

Connect your existing build, deployment, management, monitoring, and collaboration platforms. From Puppet to Chef, HipChat to Slack, ServiceNow to JIRA, Splunk to New Relic and beyond, hand off data between systems to engage the right people.

Connect with xMatters.

 

Author Comment

by:Pall Palsson
ID: 24420731
Hi, both fields (ColA and ColB) are decimal(38, 20)

If you look at the results  the upper line is the result from the Columns and the lower from just typing in the values in the select.

If I use + or - i.e. A+B or A-B I get the correct amount of decimals.  If I try to use Cast or Convert to decimal(38, 20) I Get the same result 0.710865 (0.71086500000000000)

If I multiply or divide the cutoff happens.

So neither of your proposals work.
1.00000000000000000000  1.40673489600528227000  0.710865
1.00000000000000000000  1.40673489600528227000  0.71086599389814597

Open in new window

0
 
LVL 4

Expert Comment

by:TimSledge
ID: 24420902
This should result in the same number of decimal places...

SELECT A,B,convert(decimal(38,20), A/B)  FROM
(
     SELECT colA, colB FROM thetable
) C
            
SELECT A,B,convert(decimal(38,20), A/B)  FROM
(                        
     SELECT            1.00000000000000000000 AS A, 1.40673489600528227000 AS B
) C
0
 

Author Comment

by:Pall Palsson
ID: 24421057
Tim:

What you propose results in 0.71086500000000000 which is not the same as 0.71086599389814597

Thus it doesnt work
0
 
LVL 31

Expert Comment

by:RiteshShah
ID: 24421070
what about this one?


SELECT A,B,convert(decimal(38,20), A)/convert(decimal(38,20), B)  FROM
(
     SELECT colA, colB FROM thetable
) C
     
0
 
LVL 4

Expert Comment

by:TimSledge
ID: 24421550
The code below tells the equation to limit B to 2 decimal places and should create the same result.   You actually need to do this for A as well.  The only way you can get the same result is to use the same number of decimal places for B each time and for A each time. This is back to my original response.

SELECT A,convert(decimal(38,2), B),convert(decimal(38,20), A/B)  FROM
(
     SELECT colA, colB FROM thetable
) C
           
SELECT A,convert(decimal(38,2), B),convert(decimal(38,20), A/B)  FROM
(                        
     SELECT            1.00000000000000000000 AS A, 1.40673489600528227000 AS B
) C
0
 

Author Comment

by:Pall Palsson
ID: 24423762
Hi,

I have tried to to use convert(decimal(38,20) AND cast like this X() represents what i tried to convert/cast

X(A)/B
X(A)/X(B)
A/X(B)
X(X(A)/X(B))
X(A/X(B))
X(X(A)/B)

with no success.

However QPR's solution of using float comes closest to yielding the correct result.
cast/convert   result
correct:       0.71086599389814597
float:         0.71086599389814600
decimal:       0.71086500000000000
noone:         0.710865

Open in new window

0
 
LVL 4

Accepted Solution

by:
TimSledge earned 500 total points
ID: 24423868
I just realized you need to use  Convert(Decimal(32,1) to get the same results.  When you use anything higher than 1 in the 2nd position (controls how many places to the right of the decimal ) you are not multiplying by the same numbers.

You can never get the same answer when multiplying by 1.4 one time then 1.40673489600528227000 another time.  

Convert(Decimal(TotalDigits,DigitsToRightOfDecimal) ItemToConvert) is a way to make the two multipliers the same.  Convert(Decimal(32,1), 1.4) and Convert(Decimal(32,1), 1.40673) will both multiply by 1.4.

If you go one step higher (32,2), you will be multiplying by 1.40 and 1.41 (1.406 rounded to two places).

Try the earlier suggestion, but use 32,1 instead of 32,2.
0

Featured Post

[Webinar] Learn How Hackers Steal Your Credentials

Do You Know How Hackers Steal Your Credentials? Join us and Skyport Systems to learn how hackers steal your credentials and why Active Directory must be secure to stop them. Thursday, July 13, 2017 10:00 A.M. PDT

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Long way back, we had to take help from third party tools in order to encrypt and decrypt data.  Gradually Microsoft understood the need for this feature and started to implement it by building functionality into SQL Server. Finally, with SQL 2008, …
In SQL Server, when rows are selected from a table, does it retrieve data in the order in which it is inserted?  Many believe this is the case. Let us try to examine for ourselves with an example. To get started, use the following script, wh…
In this video, viewers are given an introduction to using the Windows 10 Snipping Tool, how to quickly locate it when it's needed and also how make it always available with a single click of a mouse button, by pinning it to the Desktop Task Bar. Int…
This is my first video review of Microsoft Bookings, I will be doing a part two with a bit more information, but wanted to get this out to you folks.

719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question