What success stories leave out

A successful business story can show how something worked. To estimate how often it works, you also need to know about the attempts that never became stories.

A founder can describe a customer problem you had overlooked, a difficult negotiation or an operating change that made a company viable. The difficulty begins when a collection of successful examples starts to stand in for the full field of attempts.

Eight successful founders borrowed money. Does that make borrowing a promising route? The eight stories alone cannot answer. How many other founders borrowed, and what happened to them?

The same ten stories, two different populations

Consider a fictional collection profiling ten companies. The editor includes every company in a defined group that is still operating after two years and whose second-year sales are at least twice its first-year sales. Eight of those ten companies used borrowed startup money; two used only their founders' money. For this example, each company belongs to one of those two financing groups.

Here are two invented populations that could produce exactly that collection. These counts are constructed to expose the missing information. They are not business statistics.

Population one: borrowing was common

• Two hundred companies began with borrowed money. Eight met the editor's condition: 4 percent.
• Ten companies began with only their founders' money. Two met the condition: 20 percent.

Population two: borrowing was unusual

• Ten companies began with borrowed money. Eight met the condition: 80 percent.
• Two hundred companies began with only their founders' money. Two met the condition: 1 percent.

Both populations contain 210 companies. Both produce ten qualifying stories, eight involving borrowing. Yet the fractions within the financing groups point in opposite directions. Knowing that eight of the ten selected founders borrowed tells you nothing about which of these populations supplied the stories.

Even the complete counts would leave important questions unanswered. The companies may differ in their markets, owners, resources and reasons for borrowing. Doubling sales while remaining open does not establish profitability or a satisfactory return to the owner. The constructed comparison illustrates selection and missing denominators; it does not show that either financing method caused the outcome.

Ask how the examples became visible

A denominator is the full group against which a count is compared. Eight qualifying companies out of ten borrowed-money attempts means something very different from eight out of two hundred. When you hear only from the companies that qualified, the attempts outside the collection remain unknown.

Jerker Denrell's 2003 paper, Vicarious Learning, Undersampling of Failure, and the Myths of Management, examines this problem. The paper describes how leaving failures out of the observed population can make risky practices appear associated with performance even when they have no such relationship across the full population.

Selection can happen before anyone sits down to write. A failed operation may have little public record. A surviving founder may still be available to interview. An editor may deliberately seek unusually rapid growth. Each choice changes what the reader can learn from the resulting set.

Begin with the collection's own purpose. Is it explaining a particular life, studying a defined group or presenting exceptional outcomes? What counted as success? Were unsuccessful attempts eligible to appear? If there is no complete population behind the accounts, mark the overall success rate unknown. An engaging account does not owe you a statistic it never claimed to provide.

Keep the mechanism; examine its conditions

Infinite Upside offers a useful way to read a particular success closely. In Chapter 6, the value recognized in the acquired vehicles mattered. So did the prices buyers would pay, the time needed to sell and the speed at which operating changes took effect. The later reflection keeps the achievement and examines how little room the timetable allowed.

The reader can learn from that sequence without assuming the same timetable would work in another market. Before applying it elsewhere, the next question concerns the actual sale dates, usable proceeds and obligations of the new situation. Admiring the judgment does not settle those facts.

Chapter 13 brings a related concern into a growing company. Early customers may be unusually convenient to serve. A small test can reveal something important while leaving the economics of a larger operation unresolved. In each case, the useful lesson includes the conditions that made it possible.

When a story gives you an idea, write down the action you want to borrow and the condition it needs. Then identify what evidence is missing for your own decision. If the appeal depends on a claim about how often something succeeds, look for a defined comparison group and comparable outcomes. If the appeal is a practical mechanism, examine whether your operation can actually supply its requirements.

A story can change what you notice before it changes what you decide. Sometimes its most valuable contribution is a question you would otherwise have failed to ask.