Solved

How to reconcile multiple range estimates

Posted on 2011-03-04
13
665 Views
Last Modified: 2012-05-11
If two software engineers each give me a range estimate for the cost of a software project, say
1) $250K - $260K
2) $100K - $500K
both with a 90% confidence level, how should I reconcile this data to give me a single 90% range estimate?
0
Comment
Question by:Milewskp
  • 5
  • 3
  • 2
  • +2
13 Comments
 
LVL 10

Expert Comment

by:Martin_J_Parker
ID: 35036617
With a range like $100K-$500K the engineer has obviously decided that it's going to be about $250K and covered his arse by giving a wide margin either side, which he's 90% certain that he won't go over, even if the project gets seriously delayed.  He's taking the piss a bit but he won't get accused of underestimating the costs.

For a $250K-$260K range the other engineer either has a very good understanding of the costs and can guarantee no project slippage (very unlikely) - or he's an idiot.  Even delivering a few weeks late can drive a project way over $10K extra.

From experience of software projects I'd say that the $100K-$500K estimate would be less likely to be wrong, however the $100K figure is likely to be low, given the other engineer's estimate.

If you can think of any reasonable sized software project that has been delivered on time, within budget and to specification then you've found something extremely unusual.  Certainly no government IT project that I can think of has managed all 3 criteria.  Actually I'd be surprised if you can find a government project which hit 2 of the 3!

Given all that, the only figures I'd be confident in are the wider range.
0
 
LVL 27

Assisted Solution

by:aburr
aburr earned 250 total points
ID: 35036858
AS shown above you cannot reconcile them
However if you assume (a BIG assumption ) that both estimates are the result of normally distributed independent measurements you can treat the numbers as representing two normally distributed random variables and add the distributions.
0
 
LVL 1

Author Comment

by:Milewskp
ID: 35037459
Folks,
I work in the Project Controls group of my company, and there are/ will be situations where we get multiple estimates for all kinds of projects (IT and other).  I need a way to calculate an aggregate range estimate given the range estimates we are provided.

Now, when someone givens me a range estimate with a 90% confidence level, I make the assumption* that 90% of that individual's 90% confidence level estimates will be correct (ie, the actual values will fall within the ranges 90% of the time). My job is not to second-guess their estimates, but to provide management with the 'best' range estimate given the inputs I've received.

*You may argue that this is an unrealistic assumption, but I would direct you to Doug Hubbard's book "How to Measure Anything".  Anyway, whether this assumption is realistic or not is not the question. The question is how to calculate an aggregate range estimate.

Any help you could provide would be much appreciated.
0
 
LVL 10

Expert Comment

by:Martin_J_Parker
ID: 35037624
If you are going on a 90% rule I think you can also assume that there will be 10% of individuals whose 90% confidence levels are hopelessly wrong 90% of the time - either they don't have the experience or skills to get the numbers right.  I have seen this in practice.  Out of the approx 20 people that I work with a couple of them will consistently seriously under-estimate how long a programming task will take to do, even though they have been programming for 30+ years!

When the estimates are as far apart as the original examples I think you can reasonably assume that one of them is not right and the job is then to work out which is wrong and why.
0
 
LVL 37

Accepted Solution

by:
TommySzalapski earned 250 total points
ID: 35038561
If you assume they are normal distributions, then the 250-260 is N(255, 3.0398) and the second is N(300, 121.5914) So the variances are 9.2404 and 14784.46

So if you are adding these, it makes X~N(555, 121.629) which has a 90% confidence interval of
355k - 755k

If you are averaging them, then you would have X~N(277.5, 86.004) or a 90% conf of 136k - 419k
0
 
LVL 1

Author Comment

by:Milewskp
ID: 35038836
Hi Tommy,
<If you are averaging them, then you would have X~N(277.5, 86.004) or a 90% conf of 136k>
So you are saying that, if I assume noraml distributions, adn I assume that the estimators are correct 90% of the time (ie, the confidence interval really is 90%),  then I can assume that the cost in this example will be in the range $355k-755k 90% of the time?
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 27

Expert Comment

by:aburr
ID: 35039110
I repeat
However if you assume (a BIG assumption ) that both estimates are the result of normally distributed independent measurements you can treat the numbers as representing two normally distributed random variables and add the distributions.
0
 
LVL 37

Expert Comment

by:TommySzalapski
ID: 35040133
I agree that those are rash assumptions, but yes. Assuming that those things are true, that is how the math works out.
0
 
LVL 1

Author Comment

by:Milewskp
ID: 35041166
Hi Tommy,
<Assuming that those things are true, that is how the math works out. >
But, given our assumptions, we already know that the actual cost will be between $250K - $260K  90% of the time, so how can they also be between $136k - $419k  90% of the time? It can't be both.
0
 
LVL 27

Expert Comment

by:aburr
ID: 35041564
If indeed "we already know that the actual cost will be between $250K - $260K" why are you paying any attention to engineer 2?
0
 
LVL 1

Author Comment

by:Milewskp
ID: 35043370
Hi all,
Ignore my last post (brain hiccup).
0
 
LVL 1

Author Comment

by:Milewskp
ID: 35043385
Thanks everyoen for your input,
I'll split points between aburr (answered first), and Tommy (detailed answer) since this is a consensus question.
0
 
LVL 1

Expert Comment

by:ifpugnacious
ID: 35056870
Whole lot of questions:

1. Project efforts are more likely to be approximately log-normally distributed than normally distributed. Determining the variance of a sum of log-normally distributed variables is much messier than determining the variance of a sum of normally distributed variables, but given the slop in these numbers I wouldn't worry about it too much. Use an approximation. "Why measure it with calipers if you're going to cut it with an axe?"

2. Were these price quotes? They were stated in units of currency, after all. If so, they're really not estimates, but proposed commitments, which is a very different thing. Saying $250-$260K amounts to a fixed bid.The two engineers may have estimated the effort as the same, but the second one is much more willing to assume the risk of running over in exchange for pocketing the extra if s/he comes in under.

Much pain and suffering has occurred in our "profession" due to lack of understanding of the difference between a Target, an Estimate, and a Commitment, and the failure to compare any of them to past reality.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Decrease number proportionately 20 84
Sample Space 12 38
Triangles - computing angles 7 38
Simple Calculation for Value of Availablity 5 65
If you are using Scrum Framework or another agile process, a retrospective may be part of it. Does your team perform retrospectives? Are you getting value from your retrospectives? I see a common anti-pattern when people conduct a retrospective f…
Transparency shows that a company is the kind of business that it wants people to think it is.
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now