A/B Split Test to Achieve Your Goals

Published on
10,438 Points
5 Endorsements
Last Modified:
Community Pick
Most people's websites have a purpose. It could be to sell products as with Amazon, it could be to be to sign up community members like Digg does or could be to generate sales leads like Toyota's Build Your Own Toyota.
But how do you create web pages that are the best at accomplishing these desired outcomes as possible? This article's aim is to make you aware of A/B split testing and to inspire you to think of ways to apply split testing to your own website in order to improve its performance.

Modern technology allows internet marketers can perform split tests to compare the performance of web pages with desired outcomes such as sales, registrations or leads. Most test-savvy marketers will call these desired outcomes, no matter what they are, success events. Being able to measure performance is a major improvement over making decisions based on your gut, which as it turns out isn't always right.

In order to perform a split test, you will need a couple of important things.

The ability to serve two different versions of a page - You'll have to check with your engineers on this one; I'm just a marketing guy.
Enough traffic to reach statistical significance - You may need more or less traffic to reach statistical significance depending on the difference in performance of each of the two versions.
The ability to track the performance of each version of the page - This will probably take the form of a web analytics tool such as Omniture SiteCatalyst or Google Analytics.

Let's use an overly simplified example of a split test. Imagine that you work in the marketing department at Five Aces Poker where visitors can register and play poker for free. You've been tasked with increasing the number of new registered members. One way to do this is to improve the conversion rate (registrations divided by page views) of your registration page via split testing. Assuming that your traffic to the registration page remains constant, increasing the conversion rate will also increase the number of registrations.

Imagine that the current registration page looks like this:

 Control Registration Page
Testers usually refer to the existing version of the page as the control and any new versions as treatments.

You think that you can improve on the control page in a number of ways. As it is, the control does not encourage registration, and it has too many fields -- most of which are unnecessary. You've created the following treatment to test against the control.

 Treatment Registration Page
Split test to find out which of these pages is the best at registering members at Five Aces Poker.

Divide traffic hitting the registration page to the two different versions and track the results. The conversion rate (registrations divided by page views, remember?) is your best way to compare the performance of each version; obviously the higher the conversion rate the better.

When comparing the conversion rates, it's important that you have a large enough sample size to statistically support your conclusion. You can either crack open that old statistics book that has been sitting on your bookshelf since you graduated college, or you can cheat and use a split testing calculator like the one at PickAGuru (it's not the most beautiful thing you've ever seen but it works well). What you're looking for is the interpretation that one or the other version of the registration page is the clear and undisputed winner with a likelihood of something higher than 90% or 95% depending on your aversion to being wrong. I prefer to be at least 95% confident.

Once you've identified which version is better, you can send all of your traffic to the statistically better registration page and reap the benefits of more registrations per page view and/or you can begin your next test. Ongoing testing ensures that you are always improving your pages and doing a better job of accomplishing your objectives.

Ask questions about what you read
If you have a question about something within an article, you can receive help directly from the article author. Experts Exchange article authors are available to answer questions and further the discussion.
Get 7 days free