Books about controlled business experiments

Your team changed something and the results improved. You want to understand what that improvement establishes before making the change permanent. Choosing a book about controlled experimentation is a different task from collecting more ideas for a customer survey.

Start with Infinite Upside for the limits of what a business trial has shown. Experimentation Works examines experimentation as an organizational capability. Trustworthy Online Controlled Experiments moves into the technical subject of A/B testing. Choose by the part of the comparison you want to study.

1. Infinite Upside: keeping the test beside the proposed commitment

   By Vijay Sheth.

   Chapter 13 considers an expansion trial using spare capacity in an existing operation. Paying customers and workable delivery costs establish useful facts about that arrangement. They do not automatically establish the economics of a separate location with its own resources.

   The account also examines whether early customers resemble later ones and whether another increment of growth requires a different cost structure. The owner needs to understand what the evidence covers before relying on it.

   Read the memoir first if an encouraging result has begun to stand for a larger conclusion. It gives the experiment reading an operating purpose: the next commitment should reflect what has actually become better understood.

   The trial is not presented as a randomized A/B experiment. The memoir supplies context for asking about evidence, rather than technical instruction in controlled experimentation. The next two books address that field more directly.

2. Experimentation Works: studying the company's experimental capability

   By Stefan Thomke.

   Harvard Business Review Press's 2020 Experimentation Works: The Surprising Power of Business Experiments addresses disciplined experiments across online and physical businesses. Its stated scope includes what makes an experiment useful and how organizations build an experimentation culture.

   Choose Thomke when the management arrangement is part of your question. The company may have people proposing tests without a clear shared understanding of what counts as learning from them.

   Imagine a fictional booking business whose departments change offers independently. Each describes its own results as encouraging, yet nobody can explain the comparisons being made. The leader wants to study how experimentation becomes an organizational capability. This is an editorial example, not a case from the book.

   Its breadth is the reason to investigate the title. Publisher descriptions of successful companies do not establish an available gain for this fictional business. Investigate the management approach with the fictional company’s actual comparisons and decisions in mind.

3. Trustworthy Online Controlled Experiments: studying the technical comparison

   By Ron Kohavi, Diane Tang and Ya Xu.

   Cambridge's 2020 Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing covers online experimentation, metrics, the trustworthiness of results and statistical issues in practice.

   Choose this more technical direction when you want to examine how a controlled online comparison works. The publisher identifies material for people starting experiments as well as deeper topics for practitioners.

   The fictional booking leader might instead want to understand why two reported improvements are not automatically comparable. The relevant study could concern what was measured and which assumptions the analysis requires. That is a different commitment from a broad introduction to testing a business idea.

   The digital setting and technical depth matter. An online experiment reference is a less direct starting point for investigating a one-off building project or learning through an initial customer conversation. Examine a sample for the level you want before choosing it as your first experimental-methods book.

Choose the experimental question

Choose owner context for the commitment, Thomke for the organizational capability or Kohavi, Tang and Xu for the technical online subject. Name what you cannot yet explain about the comparison before choosing the next sample.

The business-idea selection covers customer conversations and wider validation choices. The measurement selection considers what the business should try to understand and measure.

Read Infinite Upside for an owner's account that keeps a promising trial connected to the decision it can inform. Try the opening passage before continuing into the later expansion discussion.