Fighting estimation fatigue - Part 5 (final): Best practices of software estimation

Finally we're at the point where we now know why we develop estimation fatigue while asked to estimate a software project and also we've visited some of the options that we have to avoid that fatigue (refer to the previous posts: Fighting Estimation Fatigue Part 1, Part 2, Part 3 and Part 4). But If all of those seems too much for you, and if you have a project in hand, you can just go over this post and use it as a jump starter for your project and can't wait to read through all those posts. The best practices described here can be used with the use of any specific standard software estimation model or no model at all:

1. Estimate the effort in terms of scenarios i.e. Best Case and Worst Case scenario
2. Create the estimations in terms of range value rather than a single number
3. When the project has lot of uncertainty and unknowns, provide multiple estimations assigning the uncertainties and unknowns with each estimation
4. Use a confidence level based estimation when you’re hard pressed to give a single estimation point. E.g. If you’re given no chance but told to deliver the software before the Christmas day, then the estimation could be given as – Release date on 12/25/2014 (60% confidence level)
5. Ball park figure is a dangerous trap, never fall into that. If you'are forced to give a ball park figure estimation, never forget to add the underlying assumptions and associated risks along with that estimation
6. Trust your intuition, never fear to use Estimation by Analogy technique. Though Estimation by Analogy may seem dangerous but if the organization takes similar projects repeatedly, be bold to take that route. This is not less accurate than other standard estimation methods, nevertheless the quickest and cheapest of all
7. Empirical estimation models needs time to be matured and predictable. If the software project is one time endeavor and may not be repeated, do not use empirical estimation model. The promise of improving the accuracy can’t be harvested without the repetition
8. Estimation should be done at the most possible granular level and then roll up to the complete project level to get the project estimation. The granular level estimation offsets most of the inaccuracies  in estimation
9. Estimation at granular levels can be used to defend your estimation as well. Management tend to  raise question to a lump sum estimation of a project but would think twice to question a granular level of each features that aggregate into the project level estimation. Moreover, when you will be asked to add a new feature in the software (and it is almost guaranteed that you would be asked for it), that granularity helps to identify feature(s) that you can trade in, otherwise you would have no choice but to swallow the change in to your project
10. Revisit the estimation once the project is completed. This time, re-estimate the relative size of each tasks as well as the actual effort hours and record them. Once you have enough empirical data, you can use them to provide a ball park figure when asked by executive management.

It is as important to communicate the estimation right as the estimation process itself. Often time a great deal of time and money are invested for the estimation process whereas leaving the the presentation of the estimation unfocused. It's also very important to set the right tone that creates the environment of acceptance. I've compiled some of the communication best practices for software estimation that, if followed appropriately, can increase the chance of acceptance of your estimation:

1.  Don’t fall in the trap of telling what the project sponsors and executives want to hear. Because you are the one who one who would be held responsible for the estimation commitment that you made
2. Document all the assumptions that were made throughout the estimation process. For example, if the estimation value is converted into time duration, make sure to document what was the competency level of programmers that was assumed to get the person day
3. Educate your stakeholders about the “Cone of uncertainty” of software estimation process. The stakeholders will be more susceptible to accept the higher error margin when the estimation is made early in the project's life cycle, as shown in the "cone of uncertainty"
4.  Always present the estimation in terms of a range value rather than one single number. By definition, the word estimation is a rough calculation. So it wouldn’t be a wise idea to present the estimation as a single number which would mislead the audience about it's perceived accuracy
5. If absolutely necessary to provide a single value estimation, use the plus-or-minus qualifier e.g. rather than telling that the software would take 6 months, express it in 6 months +- 2 months. So, the stakeholders gets what they wanted, and the estimator is covered from future accusation
6. You can also use a confidence factor while presenting your estimation. As an example, It’s better to communicate that you're, for instance, 70% confident that the project would be completed in 6 months i.e. if the project is targeted to deliver after 6 months, there's a 30% chance of missing the deadline
7. Don’t pretend the all sunny day scenario for your project. Though some people may consider it as pessimistic approach, but communicate the predicted implementation risks Add the perceived risks that were identified during the estimation process and use them to come up with plus-minus qualifier, range and confidence level based single estimation
8. Even though it may not be asked, it's better to present more than one estimation values with different set of scopes. As for an example: if you have a system to develop that has100 use cases, you may provide two estimations: one for the entire set i.e. 100 use cases and another is for top 80% use cases. It would help you to avoid the trap of "everything is absolutely required" statement. In reality they are always not true. So if the estimation of everything seems too long to the the stakeholders, there's an alternate with reduced set of features which is analogous to the options offered by your GPS - shortest distance or shortest time; and you do the pick

There's a wise saying that "we don't fail a project but we fail to estimate it right". I hope, after going through all these posts of the estimation series, you would have better estimate your software projects thus achieve a higher rate of successful software projects.

Fighting estimation fatigue - Part 5 (final): Software estimation best practices

Finally we're at the point where we now know why we develop estimation fatigue when  asked to estimate a software project and also we've visited some of the options that we have to avoid that fatigue (refer to the previous posts: Fighting Estimation Fatigue Part 1, Part 2, Part 3 and Part 4). But If all of those seems too much for you, and if you have a project in hand, and can't wait to read through all those posts then you can just go over this post and use it as a jump starter for your project. The best practices described here can be used with the use of any specific standard software estimation model or no model at all:

1. Estimate the effort in terms of scenarios i.e. Best Case and Worst Case scenario and then take the average of both to set the target date of your project (refer to PERT)
2. Create the estimations in the form of range value rather than a single number estimation. After all, the estimation is an educated guess, so, you don't need to pretend that you have psychic power to reveal a golden number
3. When the project has lot of uncertainty and unknowns, provide multiple estimations with different set of assumptions for each estimation
4. Use a confidence level based estimation when you’re hard pressed to give a single estimation point. E.g. If you’re given no chance but told to deliver the software before the Christmas day, then the estimation could be given as – Release date on 12/25/2014 (60% confidence level)
5. Ball park figure is a dangerous trap, never fall into that. If you're forced to give a ball park figure estimation, never forget to add the underlying assumptions and associated risks along with that estimation
6. Trust your intuition, never fear to use Estimation by Analogy technique. Though Estimation by Analogy may seem dangerous but if the organization takes similar projects repeatedly, be bold to take that route. This is not less accurate than other standard estimation methods, nevertheless the quickest and cheapest of all
7. Empirical estimation models needs time to be matured and predictable. If the software project is one time endeavor and may not be repeated, do not use empirical estimation model. The promise of improving the accuracy can’t be harvested without the repetition
8. Estimation should be done at the most possible granular level (also known as Bottom up Estimation) and then roll up to the complete project level to get the project estimation. The granular level estimation offsets most of the inaccuracies  in estimation
9. Estimation at granular levels can be used to defend your estimation as well. Management tend to  raise question to a lump sum estimation of a project but would think twice to question a granular level of each features that aggregate into the project level estimation. Moreover, when you will be asked to add a new feature in the software (and it is almost guaranteed that you would be asked for it), that granularity helps to identify feature(s) that you can trade in, otherwise you would have no choice but to swallow the change in to your project
10. If you choose to use a standard software estimation model, use at least two models and compare the results. If they're close, it would give you the confidence on your estimation
11. Revisit the estimation once the project is completed. This time, re-estimate the relative size of each tasks as well as the actual effort hours and record them. Once you have enough empirical data, you can use them to provide a ball park figure when asked by executive management.

It is as important to communicate the estimation right as the estimation process itself. Often time a great deal of time and money are invested for the estimation process whereas leaving the the presentation of the estimation unfocused. It's also very important to set the right tone that creates the environment of acceptance. I've compiled some of the communication best practices for software estimation that, if followed appropriately, can increase the chance of acceptance of your estimation:

1.  Don’t fall in the trap of telling what the project sponsors and executives want to hear. Because you are the one who one who would be held responsible for the estimation commitment that you made
2. Document all the assumptions that were made throughout the estimation process. For example, if the estimation value is converted into time duration, make sure to document what was the competency level of programmers that was assumed to get the person day
3. Educate your stakeholders about the “Cone of uncertainty” of software estimation process. The stakeholders will be more susceptible to accept the higher error margin when the estimation is made early in the project's life cycle, as shown in the "cone of uncertainty"
4.  Always present the estimation in terms of a range value rather than one single number. By definition, the word estimation is a rough calculation. So it wouldn’t be a wise idea to present the estimation as a single number which would mislead the audience about it's perceived accuracy
5. If absolutely necessary to provide a single value estimation, use the plus-or-minus qualifier e.g. rather than telling that the software would take 6 months, express it in 6 months +- 2 months. So, the stakeholders gets what they wanted, and the estimator is covered from future accusation
6. You can also use a confidence factor while presenting your estimation. As an example, It’s better to communicate that you're, for instance, 70% confident that the project would be completed in 6 months i.e. if the project is targeted to deliver after 6 months, there's a 30% chance of missing the deadline
7. Don’t pretend the all sunny day scenario for your project. Though some people may consider it as pessimistic approach, but communicate the predicted implementation risks Add the perceived risks that were identified during the estimation process and use them to come up with plus-minus qualifier, range and confidence level based single estimation
8. Even though it may not be asked, it's better to present more than one estimation values with different set of scopes. As for an example: if you have a system to develop that has100 use cases, you may provide two estimations: one for the entire set i.e. 100 use cases and another is for top 80% use cases. It would help you to avoid the trap of "everything is absolutely required" statement. In reality they are always not true. So if the estimation of everything seems too long to the the stakeholders, there's an alternate with reduced set of features which is analogous to the options offered by your GPS - shortest distance or shortest time; and you do the pick

There's a wise saying that "we don't fail a project but we fail to estimate it right". I hope, after going through all these posts of the estimation series, you would have better estimate your software projects thus achieve a higher rate of successful software projects.

Fighting estimation fatigue - Part 4: Choose the right estimation model

The availability of vast number of software estimation models sometimes make it confusing for the project manager to pick the right estimation model. Moreover, not every one wants to be expert in estimation techniques and models. If you have enough budget allocated for planning phase of your project (I doubt you would often get that in any software project), you can hire an estimation expert who would have detail expertise on various software cost estimation models and pick the right model for your project. But for those who don't want to invest their time on going through each of the models (some of which I briefly explained in the post Introduction to standard estimation models and methods), I've created a sample matrix that would provide some sort of guideline to help you identify the right estimation model. At the very least, gives you starting point.

For example, if the software project uses the Object Oriented Programming technology, almost all the estimation models can be used. Now if the development methodology is taken into consideration and if it happens to use Agile Development methodology, then the options become narrow. In that case a few of the above models remains to be a pick from the matrix e.g. Use Case Point, User Story Point, and Delphi Method. Though the above framework would provide some initial guideline to narrow down the list of models, but at the end it’s up to the Project Manager and the Software development team to decide on the best fit models for their project.

If I were you, I would not settle on a single estimation model from the above matrix but try out at least two to three estimation methods for my initial estimation and if all of them comes with a close proximity of 10% - 20%, then probably you can go with any one you like. But if you find out a large gap in your initial estimations from those different methods, then you probably should be very careful while providing your commitment to your company executives. The variation in estimations tells you that you're handling a project that has more unknowns and lot of uncertainties. I believe we haven't forgotten the lesson of the cone of uncertainty.

In my final post of this software estimation series, I would cover on the communication part of the software estimation and some of the industry best practices.

Fighting estimation fatigue - Part 3: the cone of uncertainty

Software project is all about unknowns. At the beginning of a software project, the project charter takes a very simplistic view of the final product and try to estimate the dollar amount (because as we learned, the project sponsor always look at the bottom line which is the dollar cost of the project). At this stage, the larger amounts of unknown create a larger uncertainty in estimation. But as the project moves into the deeper level of planning and implementation, the more unknown becomes known, hence the uncertainty in the estimation becomes lesser compare to the previous stage. This phenomenon is described by the concept of “The Cone of Uncertainty”, originally used in the chemical industry by the founders of the American Association of Cost Engineers (now AACE International) and got wide popularity after it’s published in Steve McConnell’s famous book “Software Estimation: Demystifying the Black Art”.

Figure: The Cone of Uncertainty from www.agilenutshell.com

According to the above graph, it’s evident that the later in the project life cycle, the better the estimation. But the ‘catch 22’ of this reality is, no one would come to the one stage down towards the certainty if the initial estimation (which is bound to be inaccurate due to the high error margin) is not given at the project inception. 

So, can this cone be beaten? As Steve McConnell mentioned –“Consider the effect of the Cone of Uncertainty on the accuracy of your estimate. Your estimate cannot have more accuracy than is possible at your project’s current position within the Cone. Another important – and difficult – concept is that the Cone of Uncertainty represents the best-case accuracy that is possible to have in software estimates at different points in a project. The Cone represents the error in estimates created by skilled estimators. It’s easily possible to do worse. It isn’t possible to be more accurate; it’s only possible to be more lucky”. Now the only option is left and that is to live with that. Below are some techniques on how to deal with that reality:

• Be honest and upfront about the reality. Though it may not be taken as a positive gesture initially, but being truthful about the risk of estimating with the expectation of high accuracy. If the project sponsors can be made convinced with the reality of software project (probably by showing some of the past history of software projects within that organization), may be padding onto the final numbers may give everyone sufficient wiggle room 
•  Attach the variability with the estimation when presenting to the project sponsors. There’s absolutely no benefit to anyone in that project to surprise the stakeholders
• Actively work on resolving the uncertainty through making the unknowns known. This is the responsibility of the estimation team to force the cone of uncertainty to narrow down. Without an active and conscious actions, the narrow down of the cone, as it appears, won't happen won't happen on it's own with the progression of the project's life cycle 

Let's try to take a postmortem look on why we have this cone of uncertainty in our software projects. The single most reason of the uncertainty of the estimation is that the estimation is actually doing a prediction (or forecast) on the capabilities of software developers. Unfortunately, by nature, human behavior is unpredictable. The same person can behave differently based on the presence of surrounding factors or absent of surrounding factors. A programmer may come up with the solution of a complex programming problem in a few minutes whereas the same person may struggle to resolve a lesser complex problem in another time. So the entire game of prediction is bound to fall apart when it's trying to predict the most unpredictable nature of human psychology. So the strategy shouldn't be to try to hit the bulls-eye with a single estimation value, rather try to maximize the chance of coming close to the actual with the estimation through the use of techniques like: range value, plus-minus factor, confidence level factor etc. That's why sometime it is being said that we don't have failed projects but just failed estimations.

Though we're living with this cone of uncertainty in every software projects but somehow we were so oblivious and pretending that didn't existed. Anyway, I hope we won't be from now onwards. In my next posts, I would talk about a model selection framework that would help to identify a standard estimation model and then finally I would provide some helpful tips and techniques on how to better communicate your estimations with some confidence and industry best practices.