# How to weight multiple estimates

This question is pretty long-winded, so please bear with me...

Two people have estimated a quantity, call it xm, with results A and B. (To put a practical perspective on it, xm could be the cost of a software project or the weight of the fuel for the space shuttle.)
A and B are range estimates (a1, a2), and (b1, b2), with means am, bm, and variances av, bv, respectively.
Assume you know from controlled tests that:
- A and B are normally distributed.
- A and B have a correlation of 0.
- (a1, a2) and (b1, b2) are 90% confidence intervals (ie, there is a 90% chance that a1<=X<=a2, and a 90% chance that b1<=X<=b2).

In order to get the best estimate for xm, you want to use an average of A and B, ie.,
xm = (1-k)am + kbm
where 0<k<1.

But (and this is my question, finally), what should k be?

NOTE: Intuitively if av=bv, then k should be 0.5, but, also intuitively, if av<bv then k should be >0.5. To illustrate the last point, say (a1, a2) is (250,260) and (b1, b2) is (0,1000). Wouldn't you put more weight on am?

PS. I'm not a student, and this is not a homework question.
LVL 1
###### Who is Participating?

Author Commented:
I think Bayes Theorem is the solution here.
If you have only estimate A, then you assume 90% on the true value being within the range (a1, a2). But when you get additional information, in the form of estimate B, you should use Bayes Theorem to consolidate the two estimates.

For details on how to do this, see the book by Hubbard: How to Measure Anything (last section of Chapter 10).
0

Commented:
Not at all. If they are both actually 90% confident, then they should be weighted equally. If you were trying to find the average salary in a company and one guy made twice as much as anyone else, you still weight his the same (1/n). Just because some number is bigger doesn't mean the weight should be different.

Actually, because of the way the variance is squared, the higher variance is sort of weighted higher than the other.
0

Commented:
Now, there is one very strange point in this problem (which has been mentioned before). You say that two different estimates of the same number are both 90% accurate, but they are incredibly different. This isn't really possible which is why the intuition is wrong. Any time you try to think intuitively about an impossible situation, you get odd conclusions.
0

Author Commented:
Hi Tommy,
<If they are both actually 90% confident, then they should be weighted equally>
Consider this example:
Say the first estimate for the amount of fuel for the space shuttle is 35,000 - 36,000 lbs, and the second estimate is 1000 - 100,000 lbs.
Intuitively, if both estimators give answers that are correct 90% of the time, it's a pretty reasonable assumption that the guy with the tighter estimate is more knowledgable than the other guy (he's probably a NASA engineer to be 90% confident of such a tight estimate). My dilemma is how to quantify this.

http://www.experts-exchange.com/Other/Math_Science/Q_26866010.html ,
we know that xv (the variance of average of the two estimates) is minimized when k = av/(av+bv), so maybe this is the answer I'm looking for. It feels right, but I'm just guessing. I'm looking for a more conclusive argument.
------------------------------------------------------

<two different estimates of the same number are both 90% accurate, but they are incredibly different. This isn't really possible>
It's not the estimates that are 90% accurate, it's the confidence levels.
Consider this example: You might be 90% confident that the high for tomorrow in your town will be 70-75F; but if all I know is that you live in the USA, my 90% confidence range would likely be much wider, say 25-90F. Two different estimates of the same number, both 90% confident, but incredibly different.
0

Commented:
<Say the first estimate for the amount of fuel for the space shuttle is 35,000 - 36,000 lbs, and the second estimate is 1000 - 100,000 lbs.
Intuitively, if both estimators give answers that are correct 90% of the time, it's a pretty reasonable assumption that the guy with the tighter estimate is more knowledgable than the other guy (he's probably a NASA engineer to be 90% confident of such a tight estimate). My dilemma is how to quantify this. >

Then weight it towards the tighter estimate. K would be based on your level of confidence. In this case if there is a 90% confidence level, then K = .9
0

Author Commented:
Hi loen,
<Then weight it towards the tighter estimate. >
Makes sense to me.

<K would be based on your level of confidence. In this case if there is a 90% confidence level, then K = .9>
But both estimates have a 90% confidence level.
My question is how to calculate k that gives the 'best' estimate.  I think that before we can do that, we have to define 'best estimate'.  I think the best estimate might be the one with the tightest range (x1,x2) that has a 90% chance of including the actual value, but I'm open to suggestions.

When you say that k should be .9, what are you using as your definition of 'best estimate'?

0

Commented:
<Intuitively, if both estimators give answers that are correct 90% of the time.>
Ok, lets take a step back. If both give correct estimates 90%, then it must follow that the second guys estimate, despite a bigger range, is irrelavent and should not be considered. The only time it would be releavent if it had greater accuracy, so lets say 92% correct vs. the 90% correct. So I would express an equation something like this

k = aA - bA   - assuming the wider estimate is correct more often (exp 92% vs 90%)

xm = (1-k)am + kbm

<A and B have a correlation of 0>
This assumption would have to be disgarded since both are now giving you an estimate on teh same item.

0

Author Commented:
<it must follow that the second guys estimate, despite a bigger range, is irrelavent and should not be considered. >
Note sure if I agree that the wider estimate should not be considered at all. For example, if the two range estimates had the same width and were both 90% confident, I would assume you agree they should be weighed equally, but if one was just 1% wider should it be dismissed? What if it was .1% wider?

<k = aA - bA >
What are a and b?

<A and B have a correlation of 0. This assumption would have to be disgarded since both are now giving you an estimate on teh same item>
Sorry, I don't follow. If you and I give daily estimates for the next day's high temperature, why does that imply the two distributions are correlated?
0

Commented:
I've requested that this question be closed as follows:

Closed

for the following reason:

This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0

Author Commented:
If no objection from the others, please accept my final post as the answer.
0

Author Commented:
If no objection from the others, please accept my final post as the answer.
0

Commented:
You should be able to do that yourself now that the auto-close has been stopped.
0

Author Commented: