Solved

losing precision in a calculated column

Posted on 2010-08-17
3
262 Views
Last Modified: 2012-05-10
The below query is an example of something I'm struggling with. It produces output like the following:

2052035      10      5      2
5036064      12      5      2
4016002      16      5      3
1010001      23      5      4
5031669      13      5      2
5031163      21      5      4

As you can see from the query, the last column is a calculated column which should give the 2nd column divided by the 3rd column.

I am trying to get the last column to give the exact DECIMAL amount, not an INTEGER. This is why I am trying to cast it etc.

How can I get the exact amount, ie. a decimal type here in this column, not an int ?

We are using SQL Server 2000
select EA.ex_pi_eval_answer_school_id, 
SUM(EA.ex_pi_eval_answer_response), 
COUNT(EA.ex_pi_eval_answer_response),
CAST(SUM(EA.ex_pi_eval_answer_response) / COUNT(EA.ex_pi_eval_answer_response) AS MONEY)

from tbl_ex_ProgramImpact_eval_answers EA

where EA.ex_pi_eval_answer_ProgramObjective_id = 1

group by EA.ex_pi_eval_answer_school_id

Open in new window

0
Comment
Question by:zorba111
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 143

Accepted Solution

by:
Guy Hengel [angelIII / a3] earned 500 total points
ID: 33454940
try this, please
CAST( SUM(EA.ex_pi_eval_answer_response) as MONEY) / CAST(COUNT(EA.ex_pi_eval_answer_response) AS DECIMAL(20,4))

Open in new window

0
 

Author Comment

by:zorba111
ID: 33455011
Got it!!
This works... I remembered from somewhere that the default is to take the type of the first argument of any operator, and cast the result of an expression to that...

select EA.ex_pi_eval_answer_school_id,
SUM(CAST(EA.ex_pi_eval_answer_response AS DECIMAL)),
COUNT(EA.ex_pi_eval_answer_response),
SUM(CAST(EA.ex_pi_eval_answer_response AS DECIMAL)) / COUNT(EA.ex_pi_eval_answer_response)

from tbl_ex_ProgramImpact_eval_answers EA

where EA.ex_pi_eval_answer_ProgramObjective_id = 1

group by EA.ex_pi_eval_answer_school_id
0
 

Author Comment

by:zorba111
ID: 33455035
@angell

yours works too, also when simplified to:

SUM(EA.ex_pi_eval_answer_response) / CAST(COUNT(EA.ex_pi_eval_answer_response) AS DECIMAL(20,4))
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to leverage one TLS certificate to encrypt Microsoft SQL traffic and Remote Desktop Services, versus creating multiple tickets for the same server.
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Viewers will learn how the fundamental information of how to create a table.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question