The Agile Manifesto (2001) was introduced to state boldly how failures of traditional system development approaches could be overcome. Rather than doing something ‘agile’ as chasing the next buzz word, being agile is radically different. Changing requirements are a major cause of software cost estimating problems, and therefore embracing change sounds counter intuitive.
Old wine in new wine skins?
Kiera Conboy investigated in 2006 whether traditional cost estimation techniques are used on agile projects, and how mangers of agile development projects adhere to the traditional critical success factors cited by the traditional cost estimation literature.
Cost estimation tools, or model-based estimation techniques use data collected from past projects combined with mathematical formulae to estimate project cost. The main model-based techniques include COCOMO, QSM SLIM, PRICE-S, SEER-SEM, and ESTIMACS. These estimation models produce an estimate of the cost, effort or duration of a project based on factors such as the size and desired functionality of the system. Expertise based approaches, based on personal memory and analogies are useful when no quantified, empirical data is available.
As engagement managers we used one or more of these models ourselves, based service delivery contracts on the outcomes and translated model results into predictions for implementation dates, required team size throughout the project and budget. We believed in the models, and dared to write contracts with a fixed price / fixed date, sometimes even fixed quality criteria, always with a fixed scope as starting point. If you were lucky the change budget remained intact during the internal review sessions and challenges by the customer on the price, in which margin from or costs of this temporary operation was the main focus. Changes became a blind spot.
Cognitive dissonance and blanks also influence one’s expertise. Human memory-based techniques have flaws, since past projects can be forgotten, details confused and important factors accidentally ignored. The Delphi technique and work breakdown structure (WBS) fall under the heading of expert judgement techniques. By involving a group of experts, instead of a single source of truth the likelihood of errors occurring in the estimation process is reduced. Other techniques in the expertise-based category include top-down and bottom-up estimation.
The application of agile methods was combined with classic fixed price / fixed date contracts with a fixed scope and provisions for changes for years, but benefit from contracts with a fixed budget (from the client’s perspective), a fixed schedule (the rhythm provided by the sprint length, set by the development team) and a flexible scope (since changes, even late ones, from the client are welcomed).
Critical success factors for estimating
Conboy lists several critical success factors to ease the estimation process. I added the way agile approaches deal with these factors.
- Involve developers, customers, managers in estimation. Scrum team, consisting of Product owner, Scrum master and Development team conduct a sprint planning. They learn from past experience, repeat the process every 1-4 weeks sprint. Developers are the temporary owners of the stories they develop. Management pressure to underestimate or do window dressing is less of an issue in agile projects. Developer can discuss user stories with the product owner for clarification prior to estimating.
- Use estimate to evaluate project personnel. In the sprint retrospective meeting concluding a sprint, just prior to next sprint’s planning event, velocity and team performance is inspected and adapted to chase process improvements for the next sprints. The aim is to get better every time, not only in terms of productivity, but also on collaboration, communications, technical support, removal of waste and building on the strengths of team members. Although I’ve seen formal job descriptions modified for the application of agile, I didn’t see any Key Performance Indicators set yet on e.g. velocity or more accurate estimations.
- Finalise requirements before estimation. The Definition of Ready helps to protect the Scrum team to estimate on one-liners on sticky notes. The cone of uncertainty is still applicable.
- Make the early estimating effort simpler instead of more complex. I’m sure the poker planning technique is easier than setting up a Product breakdown structure (PBS) first for the sake of completeness, and then translate it into a Work breakdown structure (WBS) in Microsoft Project and have the software calculate completion dates, costs and resource conflicts to be solved.
What’s the point of story points? I want a cost estimate!
The concept of Story points with an eXtreme Programming (XP) background is often introduced in agile development projects to estimate time needed to implement a story compared to a baseline set by the development team itself. Story points are intended to be a way of estimating difficulty without committing to a specific time duration. Variances in team size or time on task do not affect them. Story points express size and complexity. Since story points have no relevance to actual hours, it makes it easy for scrum teams to think abstract about the effort required to complete a story. “The most egregious mistake is probably to invest too much time or too much debate into the choice of a unit for estimates, insofar as scheduling based on velocity makes this unit inconsequential.” according to Agile Alliance.
The fuzziness that encircle story points causes jumping to conclusions about pointless estimates, costs and productivity taken from my own experiences of projects in 2014 – 2016, like:
- Why can’t we just equate 8 story points to 8 hours?
- What’s new about Story points? Didn’t we use Use case points while we were forced to Rational Unified Process a decade ago?
- The customer requires us to propose in Function points against a benchmarked productivity in hours per function point, so that we get penalties for under-performance.
- We define productivity as size in hours / size in function points.
- We’re so good in estimating effort hours, that a sprint planning event is a waste of time, even though we’ve never worked in an agile team before.
Both Function point analysis, Use case point estimation as well as Story points are useful, as long as you know what you’re talking about and recognize the context in which these techniques can prove their value. Once you reach a maturity level that you and your team don’t need planning anymore, please call me, so that I can learn from your wisdom. Discussions tagged as #noestimates in my opinion often reveal professional laziness instead of expertise.
Show me the value
You may ask to the same manager who wants a cost estimate what the business value of the proposed change is. That’s what a story should be ranked against. And, rather than a cost estimate a budget is more important. How much are you willing to pay to get value for money? Will you guide us along the agile journey and tell me what our team needs to accomplish first? Will you give me feedback every two weeks and be available for instant questions along the to foster collaboration? Why should we commit ourselves to you? Being agile involves a lively, moving engagement from all involved stakeholders to create valuable and viable products and services.