Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
Solved

# How to weight multiple estimates

Posted on 2011-03-06
Medium Priority
374 Views
This question is pretty long-winded, so please bear with me...

Two people have estimated a quantity, call it xm, with results A and B. (To put a practical perspective on it, xm could be the cost of a software project or the weight of the fuel for the space shuttle.)
A and B are range estimates (a1, a2), and (b1, b2), with means am, bm, and variances av, bv, respectively.
Assume you know from controlled tests that:
- A and B are normally distributed.
- A and B have a correlation of 0.
- (a1, a2) and (b1, b2) are 90% confidence intervals (ie, there is a 90% chance that a1<=X<=a2, and a 90% chance that b1<=X<=b2).

In order to get the best estimate for xm, you want to use an average of A and B, ie.,
xm = (1-k)am + kbm
where 0<k<1.

But (and this is my question, finally), what should k be?

NOTE: Intuitively if av=bv, then k should be 0.5, but, also intuitively, if av<bv then k should be >0.5. To illustrate the last point, say (a1, a2) is (250,260) and (b1, b2) is (0,1000). Wouldn't you put more weight on am?

PS. I'm not a student, and this is not a homework question.
0
Question by:Milewskp
[X]
###### Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

• Help others & share knowledge
• Earn cash & points
• Learn & ask questions
• 7
• 3
• 3

LVL 37

Expert Comment

ID: 35046915
Not at all. If they are both actually 90% confident, then they should be weighted equally. If you were trying to find the average salary in a company and one guy made twice as much as anyone else, you still weight his the same (1/n). Just because some number is bigger doesn't mean the weight should be different.

Actually, because of the way the variance is squared, the higher variance is sort of weighted higher than the other.
0

LVL 37

Expert Comment

ID: 35046922
Now, there is one very strange point in this problem (which has been mentioned before). You say that two different estimates of the same number are both 90% accurate, but they are incredibly different. This isn't really possible which is why the intuition is wrong. Any time you try to think intuitively about an impossible situation, you get odd conclusions.
0

LVL 1

Author Comment

ID: 35048261
Hi Tommy,
<If they are both actually 90% confident, then they should be weighted equally>
Consider this example:
Say the first estimate for the amount of fuel for the space shuttle is 35,000 - 36,000 lbs, and the second estimate is 1000 - 100,000 lbs.
Intuitively, if both estimators give answers that are correct 90% of the time, it's a pretty reasonable assumption that the guy with the tighter estimate is more knowledgable than the other guy (he's probably a NASA engineer to be 90% confident of such a tight estimate). My dilemma is how to quantify this.

From my previous question (and your answer) at:
http://www.experts-exchange.com/Other/Math_Science/Q_26866010.html ,
we know that xv (the variance of average of the two estimates) is minimized when k = av/(av+bv), so maybe this is the answer I'm looking for. It feels right, but I'm just guessing. I'm looking for a more conclusive argument.
------------------------------------------------------

<two different estimates of the same number are both 90% accurate, but they are incredibly different. This isn't really possible>
It's not the estimates that are 90% accurate, it's the confidence levels.
Consider this example: You might be 90% confident that the high for tomorrow in your town will be 70-75F; but if all I know is that you live in the USA, my 90% confidence range would likely be much wider, say 25-90F. Two different estimates of the same number, both 90% confident, but incredibly different.
0

LVL 29

Expert Comment

ID: 35057475
<Say the first estimate for the amount of fuel for the space shuttle is 35,000 - 36,000 lbs, and the second estimate is 1000 - 100,000 lbs.
Intuitively, if both estimators give answers that are correct 90% of the time, it's a pretty reasonable assumption that the guy with the tighter estimate is more knowledgable than the other guy (he's probably a NASA engineer to be 90% confident of such a tight estimate). My dilemma is how to quantify this. >

Then weight it towards the tighter estimate. K would be based on your level of confidence. In this case if there is a 90% confidence level, then K = .9
0

LVL 1

Author Comment

ID: 35059249
Hi loen,
<Then weight it towards the tighter estimate. >
Makes sense to me.

<K would be based on your level of confidence. In this case if there is a 90% confidence level, then K = .9>
But both estimates have a 90% confidence level.
My question is how to calculate k that gives the 'best' estimate.  I think that before we can do that, we have to define 'best estimate'.  I think the best estimate might be the one with the tightest range (x1,x2) that has a 90% chance of including the actual value, but I'm open to suggestions.

When you say that k should be .9, what are you using as your definition of 'best estimate'?

0

LVL 29

Expert Comment

ID: 35071829
<Intuitively, if both estimators give answers that are correct 90% of the time.>
Ok, lets take a step back. If both give correct estimates 90%, then it must follow that the second guys estimate, despite a bigger range, is irrelavent and should not be considered. The only time it would be releavent if it had greater accuracy, so lets say 92% correct vs. the 90% correct. So I would express an equation something like this

k = aA - bA   - assuming the wider estimate is correct more often (exp 92% vs 90%)

xm = (1-k)am + kbm

<A and B have a correlation of 0>
This assumption would have to be disgarded since both are now giving you an estimate on teh same item.

0

LVL 1

Author Comment

ID: 35072703
<it must follow that the second guys estimate, despite a bigger range, is irrelavent and should not be considered. >
Note sure if I agree that the wider estimate should not be considered at all. For example, if the two range estimates had the same width and were both 90% confident, I would assume you agree they should be weighed equally, but if one was just 1% wider should it be dismissed? What if it was .1% wider?

<k = aA - bA >
What are a and b?

<A and B have a correlation of 0. This assumption would have to be disgarded since both are now giving you an estimate on teh same item>
Sorry, I don't follow. If you and I give daily estimates for the next day's high temperature, why does that imply the two distributions are correlated?
0

LVL 1

Accepted Solution

Milewskp earned 0 total points
ID: 35513303
I think Bayes Theorem is the solution here.
If you have only estimate A, then you assume 90% on the true value being within the range (a1, a2). But when you get additional information, in the form of estimate B, you should use Bayes Theorem to consolidate the two estimates.

For details on how to do this, see the book by Hubbard: How to Measure Anything (last section of Chapter 10).
0

LVL 29

Expert Comment

ID: 37223820
I've requested that this question be closed as follows:

Closed

for the following reason:

This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0

LVL 1

Author Comment

ID: 37223816
If no objection from the others, please accept my final post as the answer.
0

LVL 1

Author Comment

ID: 37223821
If no objection from the others, please accept my final post as the answer.
0

LVL 37

Expert Comment

ID: 37224631
You should be able to do that yourself now that the auto-close has been stopped.
0

LVL 1

Author Closing Comment

ID: 37248624
Thanks everyone for your suggestions.
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Agile and Scrum have almost become synonymous. Have you wondered what's the difference? Scrum is just one way to be Agile. It is the most popular which leads to the common confusion. Agile actually refers to a philosophy shared by group of developmeâ€¦
When we purchase storage, we typically are advertised storage of 500GB, 1TB, 2TB and so on. However, when you actually install it into your computer, your 500GB HDD will actually show up as 465GB. Why? It has to do with the way people and computersâ€¦
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to goâ€¦
Introduction to Processes
###### Suggested Courses
Course of the Month6 days, 21 hours left to enroll

#### 704 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.