[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More


A/B Split Test to Achieve Your Goals

Published on
10,308 Points
5 Endorsements
Last Modified:
Community Pick
Most people's websites have a purpose. It could be to sell products as with Amazon, it could be to be to sign up community members like Digg does or could be to generate sales leads like Toyota's Build Your Own Toyota.
But how do you create web pages that are the best at accomplishing these desired outcomes as possible? This article's aim is to make you aware of A/B split testing and to inspire you to think of ways to apply split testing to your own website in order to improve its performance.

Modern technology allows internet marketers can perform split tests to compare the performance of web pages with desired outcomes such as sales, registrations or leads. Most test-savvy marketers will call these desired outcomes, no matter what they are, success events. Being able to measure performance is a major improvement over making decisions based on your gut, which as it turns out isn't always right.

In order to perform a split test, you will need a couple of important things.

The ability to serve two different versions of a page - You'll have to check with your engineers on this one; I'm just a marketing guy.
Enough traffic to reach statistical significance - You may need more or less traffic to reach statistical significance depending on the difference in performance of each of the two versions.
The ability to track the performance of each version of the page - This will probably take the form of a web analytics tool such as Omniture SiteCatalyst or Google Analytics.

Let's use an overly simplified example of a split test. Imagine that you work in the marketing department at Five Aces Poker where visitors can register and play poker for free. You've been tasked with increasing the number of new registered members. One way to do this is to improve the conversion rate (registrations divided by page views) of your registration page via split testing. Assuming that your traffic to the registration page remains constant, increasing the conversion rate will also increase the number of registrations.

Imagine that the current registration page looks like this:

 Control Registration Page
Testers usually refer to the existing version of the page as the control and any new versions as treatments.

You think that you can improve on the control page in a number of ways. As it is, the control does not encourage registration, and it has too many fields -- most of which are unnecessary. You've created the following treatment to test against the control.

 Treatment Registration Page
Split test to find out which of these pages is the best at registering members at Five Aces Poker.

Divide traffic hitting the registration page to the two different versions and track the results. The conversion rate (registrations divided by page views, remember?) is your best way to compare the performance of each version; obviously the higher the conversion rate the better.

When comparing the conversion rates, it's important that you have a large enough sample size to statistically support your conclusion. You can either crack open that old statistics book that has been sitting on your bookshelf since you graduated college, or you can cheat and use a split testing calculator like the one at PickAGuru (it's not the most beautiful thing you've ever seen but it works well). What you're looking for is the interpretation that one or the other version of the registration page is the clear and undisputed winner with a likelihood of something higher than 90% or 95% depending on your aversion to being wrong. I prefer to be at least 95% confident.

Once you've identified which version is better, you can send all of your traffic to the statistically better registration page and reap the benefits of more registrations per page view and/or you can begin your next test. Ongoing testing ensures that you are always improving your pages and doing a better job of accomplishing your objectives.

  • 2

Author Comment

I see that you you've set it to review... I'm not sure if I need to change anything. Please give me a hint if this is waiting on me!

Author Comment

The apostrophes have been fixed. I've set this back to editor to review. Thanks WhackAMod.
LVL 15

Expert Comment

by:Eric AKA Netminder

"...I think the way I have it is right - if I left out the portion between the commas, it would read "Most test-savvy marketers will call these desired outcomes success events." Which is what I intend to say. The interjection (or whatever its called, its been a while since grammar) is just a style thing, I suppose..."

I probably wasn't being clear. Your sentence starts off "Most test-savvy marketers..." which implies that not ALL marketers call the desired outcomes "success events", correct? So... rather than bother with what the rest of the marketers call them, I suggested putting the word "but" at the beginning of "no matter what they are" -- because leaving it out made me wonder: If a desired outcome ISN'T a success event, what IS it.

I was NOT suggesting that the sentence be altered in any other way. Structurally, the commas are in the correct place, and except for the missing condition, it makes perfect sense.

Webster's also now lists "task" as a transitive verb. It still sounds like jargon to me. Unfortunately, my copy of the OED is a mile or so away, and it's not that important.


Featured Post

OWASP Proactive Controls

Learn the most important control and control categories that every architect and developer should include in their projects.

Use Wufoo, an online form creation tool, to make powerful forms. Learn how to choose which pages of your form are visible to your users based on their inputs. The page rules feature provides you with an opportunity to create if:then statements for y…
Learn how to set-up custom confirmation messages to users who complete your Wufoo form. Include inputs from fields in your form, webpage redirects, and more with Wufoo’s confirmation options.
Other articles by this author

Keep in touch with Experts Exchange

Tech news and trends delivered to your inbox every month