Creating a Culture of Experımentation
Online experiments can be a game changer when it comes to marketing and innovation. By running some 25,000 tests a year, for instance, Booking.com has transformed itself from a small start-up to the world’s largest accommodation platform. Today scaling up an organization’s experimentation capabilities is critical, but many firms struggle to do it—not because of technology but because of culture.
To break down cultural barriers, companies need to create an environment where curiosity is nurtured, data trumps opinions, any employee can launch tests, all experiments are ethical, and a new more-democratic model of leadership prevails. Ultimately, executives have to be able to confront the possibility that they are wrong daily and willing to give their people greater autonomy.
Avoid the Pitfalls of A/B Testing
Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these tests: They focus on the average, instead of looking at how a change impacts different customer segments. They forget that customers are connected and that their interactions can affect test outcomes. And they run tests for too short a period, failing to recognize that customers’ reactions can change over time. This article describes how to avoid all those traps by applying techniques that LinkedIn and Netflix have used to produce better insights.
“The Power of These Techniques Is Only Getting Stronger”
Jeremy King has spent much of the past three decades helping firms such as eBay, Walmart, and his current employer, Pinterest, use experimentation and data to improve decision-making. In this article he shares what he’s learned about the need to balance precision with serendipity, how to promote “data democracy,” and the importance of investing in the right training for employees.
The complete Spotlight package is available in a single reprint.