Decide when agile product development makes sense, and when it’s worthless

Cynefin_framework_Feb_2011-300x296Agile product development frameworks, like every other framework, method or technique, have their limitations. Promoted as being lightweighted agile attracts many. Whether or not foundational principles are corrupted, or the underlying paradigm is neglected, is less important.

Suppose you think of Scrum as a set of techniques, events and roles. You’ll discover that executing recipes (let’s call it ‘doing scrum’) will not be effective. Why? Because the world in need of best practices is a quite simple one, not the vague world in which a product owner doesn’t know exactly what he wants, the team doesn’t know exactly how to make it and for sure changes will come along, when you’re focused to get things done.

System thinking is important to understand what’s happening here. The sense-making framework Cynefin, created by professor Dave Snowden in 1999 makes sense here. Let Snowden explain the model himself:

Seeing problems through the lens of your preferred or natural management style is a pitfall. You have to realize that there are more system characterizations which require a different management style, and a different approach for product development. To watch the 71 minutes long version Dave Snowden shared at a Lean, Agile & Scrum Conference, take a coffee and your note book first.

“Taking a linear process and draw it like a circle doesn’t make it non-linear.”

Agile as an emergent approach starts in the complex world of unknown unknowns and a final solution that is obvious in hindsight only. Analysis or requirements elicitation simply to fill a backlog as first step of a mini waterfall don’t identify risks or accurately predict the solution or effort required to solve the problem that popped up in this complex environment. Experiments are needed. Inspect and adapt, refine your backlog items, repeat as necessary. Improve your process, gain from the interactions between team members, and between the team and the user(s). Only then your problem can moved to the ‘complicated’ domain and good practices, e.g. certain technology, competences can be applied to build a viable product.

Stress and resistance

Not having the perfect process in the first round or a product that fits all needs will cause stress. Inspect and adapt the process and product takes time. Yes, the short cycle between the previous inspection and adaptation reduces risks of losing (more) money which could be invested in other value-creating initiatives. It’s no surprise that the identified problems, the learning curve, and the inherent complexity will cause a demand for simplified solutions, short cuts, and patterns. Organizational change is reverted, employees are back to square one. The consultant is dismissed, the supporting manager fired. Unhappily ever after.

Where does your organization stand regarding agile?

Frederic Laloux applied Spiral Dynamics in his book Reinventing Organizations. The different organizational cultures can be tied to agile application or the meaning agile has in an organization. Your organizational color-coded culture has determines the way agile will be received and lived.

[slideshare id=34680795&doc=gdjunzuwr7o0zwvfeonx-140514113021-phpapp01]

Dave Snowden warns that system theorists, as well as the agile community, like to define the ideal end-state and engineer the bridge to close the gap between the present and future situation. Remember, that a complex world cannot be simplified. Present problems can be solved by present technology, people and processes. Reframing problems, like Mona Patel proposes in Reframe: Shift the Way You Work, Innovate, and Think, helps.

“We can not solve our problems with the same level of thinking that created them.”

You may remember Albert Einstein on this.

Yes, it makes a huge difference to preach agile, practice it in name-only or truly self-organized swarming to tacke a problem and getting support to stay in the flow. It doesn’t make sense to pretend you’re in ‘adaptive agile’ mode, while you’re actually in a results-driven agile organization. Metrics for accomplishments, the people you hire, the organizational pyramid – whether regular or a reversed one – and the (project) management methods that can be applied, will vary.

Some provoking thoughts from Snowden’s lecture:

  • if everything is transparent, no one takes risks.
  • dealing with a representative of the users i.e. product owner, instead of the real users, reveals the still strong unease to interact with real people during the development process.
  • a sprint is still sequential. Finding solutions quicker requires parallel experiments, at least one of them to solve related problems.
  • you want the water cooler stories from people, not the over-structured versions known as use cases. Micro narratives, please!

Though the Agile Manifesto’s principles are about software development only, these contain interesting, but often forgotten, hooks:

  • Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  • Business people and developers must work together daily throughout the project.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Simplicity–the art of maximizing the amount of work not done–is essential.

Scrum is one attempt. I hope the very first sentence in the Scrum guide makes more sense now: “Scrum is a framework for developing and sustaining complex products.” The framework’s definition: “A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value.” hopefully leads to an ‘aha, now I see’.

Beyond feature-based teams

One small team cannot run parallel experiments to solve complex problems. Scrum at scale, deployed as feature-based teams, with experts grouped in guilds and tribes, sounds like an organizational structure with lines or departments. Just pick Henry Mintzberg‘s classic Structrure in Fives: Designing Effective Organizations from your management books shelf.

Though swarming is used to illustrate self-organization and emergence, as well as solving numerical problems, I don’t see a match with parallelism. Parallel organizations are needed to innovate faster. A good example is the development of the atomic bomb in the Manhattan Project. Parallel enrichment processes were developed. It goes well beyond delegating tasks and sharing values. While supporting teams is needed, the consciousness of a overarching problem, call it an unknown common enemy is needed. Standing still is no option. Act, sense and respond.

It leaves us with an invitation to think of suitable metaphors, find effective and efficient ways to organize ourselves, and unlock creativity.