MESSY, EXCITING, AND ANXIETY-RIDDEN: ADAPTIVE SOFTWARE DEVELOPMENT

http://www.jimhighsmith.com/articles/messy.htm

MESSY, EXCITING, AND ANXIETY-RIDDEN: ADAPTIVE SOFTWARE DEVELOPMENT

by Jim Highsmith

Copyright 1997 by Jim Highsmith. All rights reserved.

A RENAISSANCE IN WORLD VIEW

Requirements definition needs to be viewed within the overall context of a development life cycle, and even more importantly, an overall development philosophy -- our mental model of how the world works. And that view is changing.

This changing view was illustrated by a Harvard Business Review article in which Brian Arthur culminated more than two decades of trying to convince mainstream economists that their world view, dominated by fundamental assumptions of decreasing returns, equilibrium, and deterministic dynamics, was no longer sufficient to understand reality. The new world is one of increasing returns, instability, and inability to determine cause and effect:

The two worlds . . . differ in behavior, style, and culture. They call for different management techniques, different strategies. . . . They call for different understanding [1].

The software development community has a similar dichotomy. One is represented by the more traditional deterministic development, derived from management practices rooted in nineteenth-century Newtonian physics of stability and predictability -- or in Arthur's terms, decreasing returns. As more industries move from decreasing to increasing return environments, traditional software management practices will be inadequate to meet the challenge. This article is about the second world -- unpredictable, nonlinear, and fast. These industries, such as the Internet, are like the start of a motocross race -- 50 high-powered motorcycles at the starting line and only room for three or four at the first turn 50 yards away. Traditional practices put your bike at the back of the pack, or out of the race altogether.

This article offers a different framework -- adaptive software development -- to address the issues of this second world. From a conceptual perspective, adaptive software development (ASD) is based on complex adaptive systems (CAS) theory, which Brian Arthur and his colleagues at the Santa Fe Institute have used to revolutionize the understanding of physics, biology, evolution, and economics. It is rooted in agents, self-organization, and emergent outcomes. From a practical perspective, the ASD framework is based on years of experience with traditional software development methodologies; consulting on, practicing, and writing about rapid application development (RAD) techniques; and working with high-technology software companies on managing their product development practices.

Recent articles and conference panels have illustrated these two worlds of software development, contrasting the Software Engineering Institute (SEI) method and the "Microsoft process" (there hasn't been a consensus on what to call this alternative). I would submit that the SEI approach is an example of the deterministic approach, and the Microsoft process is an example of an adaptive development approach.

...

Waterfall Life Cycle

For many years, the dominant software life cycle was the waterfall. It is characterized by linearity and predictability, with a modicum of feedback thrown in for good measure. The waterfall approach, illustrated in Figure 1, produced the bulk of legacy systems in today's organizations.

Figure 1: The waterfall life cycle.

Evolutionary or Spiral Life Cycle

Since the mid-1980s, the evolutionary life cycle has emerged, based on the pioneering work of Tom Gilb and Barry Boehm (see Figure 2). (For the purposes of this article, I won't differentiate between the two.) While the evolutionary model has moved into the mainstream, many practitioners have not changed their deterministic mind-set. Long-term predictability has been abandoned for short-term predictability, but it is predictability nonetheless. For example, Tom Gilb's recent work entails detailed component planning and great precision in specifying requirements. Some practices, such as RAD, utilize evolutionary life cycles in less deterministic ways.

Figure 2: The evolutionary life cycle.

The Adaptive Life Cycle

The adaptive model is built on a different world view. While cyclical like the evolutionary model, the phase names reflect the unpredictable realm of increasingly complex systems (see Figure 3). Adaptive development goes further than its evolutionary heritage in two key ways. First, it explicitly replaces determinism with emergence. Second, it goes beyond a change in life cycle to a deeper change in management style. The difference can be subtle. For example, as the environment changes, those using a deterministic model would look for a new set of cause-and-effect rules, while those using the adaptive model know there are no such rules to find.

Figure 3: The adaptive cycle.

ADAPTIVE SOFTWARE DEVELOPMENT

...