A/B Testing Simulation

Gain mastery in the latest A/B Testing Techniques

Before launching a new product, most organizations run A/B tests so they can make data-driven choices about their go-to-market strategy. Decisions such as which marketing message works best with consumers, what is the best way to describe the product’s benefits and features in the marketplace, and what price is the consumer willing to pay? Those who aren’t taking advantage of these fast and simple experiments are not only operating in the dark, but are missing out on potential sales and revenue.

The world of A/B testing has rapidly evolved over the past decade, with new streams of data, new methods for analysis, and new techniques for customer targeting. This simulation is designed to expose learners to key A/B testing concepts, with a focus on highlighting the importance of the explore-exploit tradeoff—a fundamental tension between acting quickly on uncertain results and gathering more data to make more precise estimates.

Rather than learning about significance testing that are not ideal for many business problems , learners are encouraged to think about A/B testing in ways that better reflect practical tradeoffs in hypothesis testing within organizations. Without requiring advanced statistics or mathematics degrees, Wharton Interactive’s A/B Testing Simulation helps learners integrate these core concepts in an engaging and interactive simulated environment.

Playing the role of the eCommerce director of Nano, a company that has just launched its next generation smartphone, learners must increase customer conversion rates to maximize profits. To evaluate what components of the Nanophone website are key for driving customer behavior, learners have access to a realistic eCommerce test suite where, over the course of 12 time periods, they run a series of experiments testing multiple variations of each component. The in-game eCommerce dashboard is modelled after real-world testing interfaces, where learners:

• set up their experiments
• gather data about customer conversion rates
• evaluate that data
• make decisions about the go to market strategy

There are two stages to the experience:

1. Practice Mode: Learners practice setting up, running, and evaluating experiments.
2. Tournament Mode: Learners compete in teams applying the strategies they learned in practice mode.

The tournament mode is a different base market from the practice mode, meaning that consumer preferences and sensitivities will be different, thus challenging learners to take the strategies they learned from the practice mode and adapt them to the new market in the tournament mode.