Carla's software testing

Test obsessed

Decisions, Decisions

@2000 Sue Peterson, www.cowgrilcoder.com

My husband bought his father's business last month. It was not an easy decision. Like most modern managers, I've been taught that the best decisions are the product of logical, deductive thinking. But we didn't have time for that. Was our decision the absolutely best one we could have made?

My husband bought his father's business last month. It was not an easy decision. Like most modern managers, I've been taught that the best decisions are the product of logical, deductive thinking. You should analyze the situation, generate and then compare all of your options, and judge the probabilities. In an ideal world, we would have reviewed the financial reports of our existing business, examined the finances of the company we were considering buying, built models of future cash flows under varying conditions, and then unemotionally followed whatever course of action would maximize our profits while minimizing our risks. But we didn't have time for that. The opportunity dropped into our lap with no prior warning. We had less than a week to make the decision and to finish all of the paperwork, or we'd miss some crucial advertising deadlines. And in our industry, no advertising means no business.

The rational judgment model

Traditional economic theory teaches what is called the "rational judgment" model. This theory holds that the best possible decisions can only be reached by comparing all of the relevant information and by statistically weighing all of your options. Decisions should be logical, deductive, and thorough. Unfortunately for the theorists, this whole process seems to have very little connection to what actually happens in real life, where information is often missing, incomplete, misleading or just plain wrong. In real life, we rarely have the luxury of all the time we need. In real life, pesky emotions intrude on the most logical of decisions, forcing us to take into account more than dry statistics and profit and loss sheets. In real life, even if we can isolate all of the relevant factors, the combinatorial explosion that occurs when we try to account for how everything affects everything else quickly makes the exercise unrealistic.

The Nobel Prize winning psychologist Herbert Simon (1957, economics) described an alternative to the rational judgment model more than 40 years ago. His theory of "bounded rationality" states that it is often impractical or impossible in real life to calculate an optimal strategy. In fact, in many situations, there may be no single best strategy. Instead, he believed that people use heuristics - simple rules of thumb based on their experience - to reach a good enough, decision.

These days, bounded rationality is gaining renewed attention in such divergent fields as psychology, economics, artificial intelligence, and animal behavior. I just finished reading the book Sources of Power - How People Make Decisions, by Gary Klein (ISBN 0-262-11227-2). Unlike most of the popular business press, which is often anecdotal with little data or rigor behind it, Sources was written by a cognitive psychologist and is backed by his many years of field research. Klein is interested in naturalistic decision-making - "the study of how people use their experience to make decisions in field settings." He says, "We all have areas in which we can use our experience to make rapid and effective decisions, from the mundane平凡的 level of shopping to the high-stakes level of fire-fighting." He wants to know how we do it and how we can get better at it.

Four mental strategies

In particular, he describes four mental strategies - sources of power he calls them - some decision-makers almost seem to pick a decision out of thin air. In reality, of course, these patterns are subtle and don't always reach a conscious level of our awareness.

Spooky or not, psychological studies have indicated that intuition is based in biology, as some brain-damaged patients lack the intuition that normal subjects show as a matter of course. Since intuition grows out of experience, it can be cultivated by deliberately seeking a wide range of experience. It can also be formally trained. If the experts in a field compile a list of the perceptual clues that they use, they can then use those lists to bring trainees up to speed quickly and to spread a common experience base amongst their peers.

Trained intuition is remarkably effective, I relied on mine a lot last month. My husband and I have owned a similar business for 20 years, so I had a gut-level feeling for what kind of cash flow, income, and outgo we could expect with the new company. Of course, intuition isn't perfect and that gave me a few nightmares while we were struggling to make our decision.

Mental simulation

Mental simulation is the natural partner to intuition. Intuition helps us see what is there, mental simulation helps us imagine what may be there in the future or what might have been there in the past. Klein defines mental simulation as "the ability to imagine people and objects consciously and to transform those people and objects through several transitions, finally picturing them in a different way than at the start." Mental simulation allows a decision-maker to mentally experiment with several different options and to try out various steps, before he chooses a course of action.

I learned to rely on my "mental movies" as I called them, long before I knew the formal term for the technique. I remember one incident in particular that happened many years ago. I borrowed a friend's horse one afternoon so I could take my husband trail riding. Smoky was gentle, but I'd recently seen him get nervous when cross-tied. I happened to look up while I was saddling another horse, to see my husband standing in front of Smoky. I was uneasy for some reason, and asked him to move. A couple of minutes later, Smoky reared without warning and went over backwards. If my husband had stayed where he was, his head would have been kicked off. As it was, I caught the horse, soothed him, and we had a nice ride. My husband still thinks I'm clairvoyant. I learned to trust my mental picture shows.

The very fluidity of a mental simulation is both its greatest asset and its greatest bane. Working with thought-stuff, we can construct simulations that would be impossibly expensive or time consuming in the real world. This is an incredible advantage when time and resources are short, but if we're not careful we can use the very power of our simulations to explain away any inconvenient facts. Nobelist Murray Gell-Mann's book, The Quark and the Jaguar (ISBN 0716727250), tells a story about two physicists who were climbing near Aspen, Colorado. When they started their descent, they got turned around and came down on the wrong side of the mountain. As they descended, they looked down and saw a lake, which they identified as the familiar Crater Lake and which they were expecting to see about then. One of the men casually mentioned that there was a dock on the lake. Crater Lake has no dock. Unfazed, his friend replied "They must have built it since we left this morning." They did make it home finally, several days later.

We can smile at the impossibility of building a dock in a morning, but these were not stupid people. Just the reverse, they were smart enough to fit any inconvenient facts into their mental simulation of where they were. Luckily for us, we can improve the way we use mental simulations. I know several software managers and consultants who like to use what they call a "pre-mortem strategy." Once a planner has created a plausible mental simulation, it is more difficult for them to imagine events unfolding in any other way. In Klein's version of the technique, the planner is asked to imagine that it is now the future, his plan has been tried and it has failed. That's all the information he has, he now has to determine why it failed. This perspective helps to knock the planner out of his overconfidence. Of course, once he starts thinking seriously about the flaws in his plan, he is in better shape to guard against them or to create a better alternative.

Software consultant and author Gerald M. Weinberg recommends something similar. He taught me that if I couldn't think of three ways my plan could fail, I hadn't really thought about it. My husband and I often perform this service for each other, as we spot the holes in each other's thinking.

Mental simulations are a conscious strategy. We usually know when we're using a simulation, and we can watch ourselves think as it unfolds in our imagination. A metaphor is equally powerful, but much subtler. When Shakespeare said, "My love is like a red rose," he framed our understanding and expectations of love. A metaphor draws on one domain to explain another. A good metaphor affects both what we are able to see in the world around us and how we interpret what we see. As an example, Klein points out how the metaphor "Arguments are like war" makes certain actions seem natural and other actions less likely. When we are under the influence of this metaphor, it's obvious that we need to attack our "opponents" and to defend our own weak points. But if we learn "Arguments are like music practice" we may be alert for ways to work together more harmoniously.

Metaphor

We use metaphors constantly in user interface and hardware design. But it's harder for us to see the metaphors that affect the ways we talk to each other or the management decisions we make every day. In the new edition of Peopleware (ISBN 0-932633-43-9), DeMarco and Lister explore how our view of teamwork changes when we shift from thinking about teamwork in sports to the harmony produced by a choir or glee club. Now, I've used the teamwork metaphor for years but I've never thought to ask myself exactly what I meant by "a team." Jerry Weinberg reports that he once asked a manager what kind of team he meant his people to emulate, and the reply was "A ski team." I'm still wondering how a software group can act like a ski team. Maybe we should all ride up the mountain together…?

Storytelling

In Peopleware, DeMarco and Lister make their point about the power of metaphor by telling a story, the story of what happened to their partnership when they took choir lessons together. By using a story to make their point, they are following a tradition that is tens of thousands of years old. Storytelling is a teaching tradition. A good story is an abstraction that filters out the unimportant factors in a situation so that we can see the important points clearly. A good story is dramatic, it has characters that we empathize with. A good story has lessons to teach us, in order to do so it must have an conclusion that resolves the conflict.

As a profession, we are fortunate to have a literature filled with good storytellers. People like Jerry Weinberg, Tom DeMarco and Tim Lister, P.J. Plauger, Henry Petroski, Robert Glass, Don Norman, and many others have been telling us their stories for decades, if we will but listen. And closer to home, well-run peer reviews and postmortems are excellent ways for a software organization to create and to pass on its own stories.

Metaphors and stories work because they "chunk" experience into smaller units. Herbert Simon believed that experts develop their skill by building a large store of remembered patterns, each individual pattern compresses and summarizes an experience. He called this their "vocabulary." As a programming manager, when I mention "Brooks Law" to a peer, we both remember what can happen when you add more people to a late project. When I ask my husband "What kind of team are we today?" we both remember to make sure that we mean the same thing when we use a term. An extensive "vocabulary" literally allows the expert to deal with more information at one time without becoming overloaded.

The power of experience

Experience is the underlying commonality in all of Klein's sources of power. Accurate intuition and pattern matching depend on our experience in the domain. Our ability to create useful simulations is based upon our knowledge and past experience. The best stories are always a distillation of personal experience, while metaphors encode group experience. Of course these aren't the only sources of power available to us. Bounded rationality should never be used as an excuse for ill-informed and impulsive decisions.

Rational analysis is a very important source of power, one that has fueled the explosive growth of the computing field and the modern world in general. Klein compares rational, analytical thinking to our foveal vision. It allows us to make very fine distinctions, distinctions that are simply impossible any other way. But if we don't use our peripheral vision at the same time, we will quickly lose track of where we are and what is going on around us.

The lesson I took away from last month is: "Trust your gut, but verify your assumptions." We decided to go ahead with the purchase, based on our gut feeling that the cash flow would be sufficient to break even the first year, and recognizing the many powerful emotions involved when a family business passes from father to son. That emotional stake was at least as important as the financial analysis in my husband's decision. But we made sure to check out as many of the assumptions we were making as we could in the week we had. And I spent the week running mental simulations of possible futures, trying to spot any potholes so we could avoid them or at the very least prepare ourselves for them.

Was our decision the absolutely best one we could have made? I don't know. I'll never know. But it was the best one we could have made at that time, under those conditions. And that's good enough for me.

Further reading Murray Gell-Mann, The Quark and the Jaguar: Adventures in the Simple and the Complex. (ISBN 0-716-72725-0)

Watts S. Humphrey, Managing Technical People (chapter 10). (ISBN 0-201-54597-7)

Gary Klein, Sources of Power -- How People Make Decisions. (ISBN 0-262-11227-2)

Herbert Alexander Simon, The Sciences of the Artificial. (ISBN 0-262-69191-4)

Bruce Bower, "Simple Minds, Smart Choices," Science News, May 29, 1999, page 348. www.sciencenews.org/sn_arc99/5_29_99/bob2.htm

The author: Sue Petersen is an anthropologist-turned-programmer-turned-manager who lives the good life in the country with her husband, two sons, and a whole passel of cats, horses, and various other wildlife. She is the Vice President, and the total IT department, for the family business she & her husband have run since 1979. You can reach Sue at: www.networkboy.com/suep/

posted on 2007-04-18 14:54  carla  阅读(239)  评论(0编辑  收藏  举报

导航