Browse All Articles > A/B Split Test to Achieve Your Goals
Most people's websites have a purpose. It could be to sell products as with Amazon, it could be to be to sign up community members like Digg does or could be to generate sales leads like Toyota's Build Your Own Toyota.
But how do you create web pages that are the best at accomplishing these desired outcomes as possible? This article's aim is to make you aware of A/B split testing and to inspire you to think of ways to apply split testing to your own website in order to improve its performance.
Modern technology allows internet marketers can perform split tests to compare the performance of web pages with desired outcomes such as sales, registrations or leads. Most test-savvy marketers will call these desired outcomes, no matter what they are, success events. Being able to measure performance is a major improvement over making decisions based on your gut, which as it turns out isn't always right.
In order to perform a split test, you will need a couple of important things.
Let's use an overly simplified example of a split test. Imagine that you work in the marketing department at Five Aces Poker where visitors can register and play poker for free. You've been tasked with increasing the number of new registered members. One way to do this is to improve the conversion rate (registrations divided by page views) of your registration page via split testing. Assuming that your traffic to the registration page remains constant, increasing the conversion rate will also increase the number of registrations.
Imagine that the current registration page looks like this:
![Control Registration Page]()
Testers usually refer to the existing version of the page as the control and any new versions as treatments.
You think that you can improve on the control page in a number of ways. As it is, the control does not encourage registration, and it has too many fields -- most of which are unnecessary. You've created the following treatment to test against the control.
![Treatment Registration Page]()
Split test to find out which of these pages is the best at registering members at Five Aces Poker.
Divide traffic hitting the registration page to the two different versions and track the results. The conversion rate (registrations divided by page views, remember?) is your best way to compare the performance of each version; obviously the higher the conversion rate the better.
When comparing the conversion rates, it's important that you have a large enough sample size to statistically support your conclusion. You can either crack open that old statistics book that has been sitting on your bookshelf since you graduated college, or you can cheat and use a split testing calculator like the one at PickAGuru (it's not the most beautiful thing you've ever seen but it works well). What you're looking for is the interpretation that one or the other version of the registration page is the clear and undisputed winner with a likelihood of something higher than 90% or 95% depending on your aversion to being wrong. I prefer to be at least 95% confident.
Once you've identified which version is better, you can send all of your traffic to the statistically better registration page and reap the benefits of more registrations per page view and/or you can begin your next test. Ongoing testing ensures that you are always improving your pages and doing a better job of accomplishing your objectives.
But how do you create web pages that are the best at accomplishing these desired outcomes as possible? This article's aim is to make you aware of A/B split testing and to inspire you to think of ways to apply split testing to your own website in order to improve its performance.
Modern technology allows internet marketers can perform split tests to compare the performance of web pages with desired outcomes such as sales, registrations or leads. Most test-savvy marketers will call these desired outcomes, no matter what they are, success events. Being able to measure performance is a major improvement over making decisions based on your gut, which as it turns out isn't always right.
In order to perform a split test, you will need a couple of important things.
The ability to serve two different versions of a page - You'll have to check with your engineers on this one; I'm just a marketing guy.
Enough traffic to reach statistical significance - You may need more or less traffic to reach statistical significance depending on the difference in performance of each of the two versions.
The ability to track the performance of each version of the page - This will probably take the form of a web analytics tool such as Omniture SiteCatalyst or Google Analytics.
Let's use an overly simplified example of a split test. Imagine that you work in the marketing department at Five Aces Poker where visitors can register and play poker for free. You've been tasked with increasing the number of new registered members. One way to do this is to improve the conversion rate (registrations divided by page views) of your registration page via split testing. Assuming that your traffic to the registration page remains constant, increasing the conversion rate will also increase the number of registrations.
Imagine that the current registration page looks like this:
Testers usually refer to the existing version of the page as the control and any new versions as treatments.
You think that you can improve on the control page in a number of ways. As it is, the control does not encourage registration, and it has too many fields -- most of which are unnecessary. You've created the following treatment to test against the control.
Split test to find out which of these pages is the best at registering members at Five Aces Poker.
Divide traffic hitting the registration page to the two different versions and track the results. The conversion rate (registrations divided by page views, remember?) is your best way to compare the performance of each version; obviously the higher the conversion rate the better.
When comparing the conversion rates, it's important that you have a large enough sample size to statistically support your conclusion. You can either crack open that old statistics book that has been sitting on your bookshelf since you graduated college, or you can cheat and use a split testing calculator like the one at PickAGuru (it's not the most beautiful thing you've ever seen but it works well). What you're looking for is the interpretation that one or the other version of the registration page is the clear and undisputed winner with a likelihood of something higher than 90% or 95% depending on your aversion to being wrong. I prefer to be at least 95% confident.
Once you've identified which version is better, you can send all of your traffic to the statistically better registration page and reap the benefits of more registrations per page view and/or you can begin your next test. Ongoing testing ensures that you are always improving your pages and doing a better job of accomplishing your objectives.
Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.
Comments (3)
Author
Commented:Author
Commented:Commented:
"...I think the way I have it is right - if I left out the portion between the commas, it would read "Most test-savvy marketers will call these desired outcomes success events." Which is what I intend to say. The interjection (or whatever its called, its been a while since grammar) is just a style thing, I suppose..."
I probably wasn't being clear. Your sentence starts off "Most test-savvy marketers..." which implies that not ALL marketers call the desired outcomes "success events", correct? So... rather than bother with what the rest of the marketers call them, I suggested putting the word "but" at the beginning of "no matter what they are" -- because leaving it out made me wonder: If a desired outcome ISN'T a success event, what IS it.
I was NOT suggesting that the sentence be altered in any other way. Structurally, the commas are in the correct place, and except for the missing condition, it makes perfect sense.
Webster's also now lists "task" as a transitive verb. It still sounds like jargon to me. Unfortunately, my copy of the OED is a mile or so away, and it's not that important.
Published.