Solved

# How to reconcile multiple range estimates

Posted on 2011-03-04
671 Views
If two software engineers each give me a range estimate for the cost of a software project, say
1) \$250K - \$260K
2) \$100K - \$500K
both with a 90% confidence level, how should I reconcile this data to give me a single 90% range estimate?
0
Question by:Milewskp
• 5
• 3
• 2
• +2

LVL 10

Expert Comment

ID: 35036617
With a range like \$100K-\$500K the engineer has obviously decided that it's going to be about \$250K and covered his arse by giving a wide margin either side, which he's 90% certain that he won't go over, even if the project gets seriously delayed.  He's taking the piss a bit but he won't get accused of underestimating the costs.

For a \$250K-\$260K range the other engineer either has a very good understanding of the costs and can guarantee no project slippage (very unlikely) - or he's an idiot.  Even delivering a few weeks late can drive a project way over \$10K extra.

From experience of software projects I'd say that the \$100K-\$500K estimate would be less likely to be wrong, however the \$100K figure is likely to be low, given the other engineer's estimate.

If you can think of any reasonable sized software project that has been delivered on time, within budget and to specification then you've found something extremely unusual.  Certainly no government IT project that I can think of has managed all 3 criteria.  Actually I'd be surprised if you can find a government project which hit 2 of the 3!

Given all that, the only figures I'd be confident in are the wider range.
0

LVL 27

Assisted Solution

aburr earned 250 total points
ID: 35036858
AS shown above you cannot reconcile them
However if you assume (a BIG assumption ) that both estimates are the result of normally distributed independent measurements you can treat the numbers as representing two normally distributed random variables and add the distributions.
0

LVL 1

Author Comment

ID: 35037459
Folks,
I work in the Project Controls group of my company, and there are/ will be situations where we get multiple estimates for all kinds of projects (IT and other).  I need a way to calculate an aggregate range estimate given the range estimates we are provided.

Now, when someone givens me a range estimate with a 90% confidence level, I make the assumption* that 90% of that individual's 90% confidence level estimates will be correct (ie, the actual values will fall within the ranges 90% of the time). My job is not to second-guess their estimates, but to provide management with the 'best' range estimate given the inputs I've received.

*You may argue that this is an unrealistic assumption, but I would direct you to Doug Hubbard's book "How to Measure Anything".  Anyway, whether this assumption is realistic or not is not the question. The question is how to calculate an aggregate range estimate.

0

LVL 10

Expert Comment

ID: 35037624
If you are going on a 90% rule I think you can also assume that there will be 10% of individuals whose 90% confidence levels are hopelessly wrong 90% of the time - either they don't have the experience or skills to get the numbers right.  I have seen this in practice.  Out of the approx 20 people that I work with a couple of them will consistently seriously under-estimate how long a programming task will take to do, even though they have been programming for 30+ years!

When the estimates are as far apart as the original examples I think you can reasonably assume that one of them is not right and the job is then to work out which is wrong and why.
0

LVL 37

Accepted Solution

TommySzalapski earned 250 total points
ID: 35038561
If you assume they are normal distributions, then the 250-260 is N(255, 3.0398) and the second is N(300, 121.5914) So the variances are 9.2404 and 14784.46

So if you are adding these, it makes X~N(555, 121.629) which has a 90% confidence interval of
355k - 755k

If you are averaging them, then you would have X~N(277.5, 86.004) or a 90% conf of 136k - 419k
0

LVL 1

Author Comment

ID: 35038836
Hi Tommy,
<If you are averaging them, then you would have X~N(277.5, 86.004) or a 90% conf of 136k>
So you are saying that, if I assume noraml distributions, adn I assume that the estimators are correct 90% of the time (ie, the confidence interval really is 90%),  then I can assume that the cost in this example will be in the range \$355k-755k 90% of the time?
0

LVL 27

Expert Comment

ID: 35039110
I repeat
However if you assume (a BIG assumption ) that both estimates are the result of normally distributed independent measurements you can treat the numbers as representing two normally distributed random variables and add the distributions.
0

LVL 37

Expert Comment

ID: 35040133
I agree that those are rash assumptions, but yes. Assuming that those things are true, that is how the math works out.
0

LVL 1

Author Comment

ID: 35041166
Hi Tommy,
<Assuming that those things are true, that is how the math works out. >
But, given our assumptions, we already know that the actual cost will be between \$250K - \$260K  90% of the time, so how can they also be between \$136k - \$419k  90% of the time? It can't be both.
0

LVL 27

Expert Comment

ID: 35041564
If indeed "we already know that the actual cost will be between \$250K - \$260K" why are you paying any attention to engineer 2?
0

LVL 1

Author Comment

ID: 35043370
Hi all,
Ignore my last post (brain hiccup).
0

LVL 1

Author Comment

ID: 35043385
I'll split points between aburr (answered first), and Tommy (detailed answer) since this is a consensus question.
0

LVL 1

Expert Comment

ID: 35056870
Whole lot of questions:

1. Project efforts are more likely to be approximately log-normally distributed than normally distributed. Determining the variance of a sum of log-normally distributed variables is much messier than determining the variance of a sum of normally distributed variables, but given the slop in these numbers I wouldn't worry about it too much. Use an approximation. "Why measure it with calipers if you're going to cut it with an axe?"

2. Were these price quotes? They were stated in units of currency, after all. If so, they're really not estimates, but proposed commitments, which is a very different thing. Saying \$250-\$260K amounts to a fixed bid.The two engineers may have estimated the effort as the same, but the second one is much more willing to assume the risk of running over in exchange for pocketing the extra if s/he comes in under.

Much pain and suffering has occurred in our "profession" due to lack of understanding of the difference between a Target, an Estimate, and a Commitment, and the failure to compare any of them to past reality.
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

We are taking giant steps in technological advances in the field of wireless telephony. At just 10 years since the advent of smartphones, it is crucial to examine the benefits and disadvantages that have been report to us.
Online collaboration can help businesses be more efficient, help employees grow their skills and foster a team environment.
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to goâ€¦
This is a video describing the growing solar energy use in Utah. This is a topic that greatly interests me and so I decided to produce a video about it.