A/B Testing Simulation

Gain mastery in the latest A/B Testing Techniques

Before launching a new product, most organizations run A/B tests so they can make data-driven choices about their go-to-market strategy. Decisions such as which marketing message works best with consumers, what is the best way to describe the product’s benefits and features in the marketplace, and what price is the consumer willing to pay? Those who aren’t taking advantage of these fast and simple experiments are not only operating in the dark, but are missing out on potential sales and revenue.

The world of A/B testing has rapidly evolved over the past decade, with new streams of data, new methods for analysis, and new techniques for customer targeting. However, students are learning about these developments in ways that are too theoretical, meaning that they aren’t developing an intuition about how to use A/B tests for real-world decision making. This simulation is designed to expose students to these concepts, with a key focus on highlighting the importance of the explore-exploit tradeoff—a fundamental tension between acting quickly on uncertain results and gathering more data to make more precise estimates.

Rather than teaching notions of significance testing that are not ideal for many business problems , students are encouraged to think about A/B testing in ways that better reflect practical tradeoffs in hypothesis testing. Without requiring advanced statistics or mathematics degrees, Wharton Interactive’s Digital Marketing: A/B Testing Simulation helps students learn about these core concepts in an engaging and interactive simulated environment.

Playing the role of the eCommerce director of Nano, a company that has just launched its next generation smartphone, students must increase customer conversion rates to maximize profits. To evaluate what components of the Nanophone website are key for driving customer behavior, students have access to a realistic eCommerce test suite where, over the course of 12 time periods, they run a series of experiments testing multiple variations of each component. The in-game eCommerce dashboard is modelled after real-world testing interfaces, where students set up their experiments, gather data about customer conversion rates, evaluate that data, and make decisions about the go to market strategy for the Nano product based on what they learned.

There are two stages to the experience, allowing students to run through a single-player Practice Mode ahead of the team-based Tournament Mode, where they’ll compete against other teams in the class. In the practice mode, students are able to familiarize themselves with the test suite. The tournament mode is a different base market from the practice mode, meaning that consumer preferences and sensitivities will be different, thus challenging students to take the strategies they learned from the practice mode and adapt them to the new market in the tournament mode.

IMPORTANT: Remember to create a class for each of your sections, and then add this game to each class