The Death of A/B Split Testing

It’s official… A/B Split Testing is Dead.

I hope you’re ready. Because today I am killing old school A/B Split Testing. But don’t worry – I’m also going to teach you what to do instead. 

Proper split testing is one of the most important skills any online marketer needs to have. Because if you split test correctly – you can make almost any campaign profitable. And by using the method that I’m about to teach you – you won’t have risk a lot of money to do it.

So please read carefully and get ready to learn.

In this post I will show you:

• How I learned to split test
• Why A/B split testing is Dead
• Why my method works better
• How to set it up in your tracking software

No landing page yet? Read: How to Launch Your First Campaign

I Use CPVLab

For this example we’ll be using CPVLab, my tracking software of choice.

One of the main reasons I use CPVLab is because it simplifies the split testing process and allows me to control the exact percentage of visitors who see my new page (which is key to the whole thing).

If you want to use CPVLab click here and use the promo code “MALAN50″ to save $50. Offer expires on March 1st so if you want it, now is the time to get it.

Note: If you’re using Prosper202, don’t leave. There are also instructions here for you too.

My First Split Test

In late 2008 I launched my first attempt using paid traffic. It was an Adwords PPC campaign on the search network with a $50/day budget.

In earlier years I’d been a web designer so I had 100s of sites under my belt. I figured the landing page part of it would be a breeze.

I spent two days crafting the perfect design. I obsessed over the details. It was beautiful. As I activated my campaign and started the traffic I knew I would succeed… I mean who could resist this amazing design.

The traffic started coming in. I got really excited… Then –

Nothing.

After my first day’s $50 budget was spent I looked at the stats in the affiliate network and realized that I’d made no sales. I then logged into my tracking software and saw that my landing page had gotten zero clicks.

Not a single click.

Confused – I went to an online marketing forum and wrote a post about what was happening. I received a single response. At first I considered it an insult.. then I realized it was the best advice anyone had ever given me.

“99% of the time the problem is not the campaign, it is the affiliate”

What he was saying was that the campaign was not failing because of a problem with the campaign. It was failing because I was the affiliate. And if I could fix “me” I could fix the campaign.

“Fixing Me” meant I needed to step up my game and split test like a super affiliate to find a winning ad > landing page > offer combination.

My super-designed landing page sucked. And I had to admit that. I was pissed. But I now understood more about how the game works.

So I Started Split Testing Like F#cking Crazy

I began rotating a fury of landing pages to the traffic until I found a design that got a few clicks. A few design changes later and it was getting a lot more clicks. Then with a few more tweaks I started getting conversions.

Then just for fun I downloaded the homepage of a blog I read, added my own content and tested it. It looked awful – but it started getting clicks AND conversions like crazy.

In about a week I went from 0% Click Thru Rate (CTR) and zero sales to a healthy 20% CTR and sales coming in. I did this by testing multiple pages against each other and then making changes, based on the data I found.

You May Want To Read: How I Figure Out What to Split Test

The 3 Rules of Split Testing:

From that first experience I realized that testing is about the most important thing I can do. And from that day forward I have 2-3 landing pages rotating at all times.

Here are the 3 rules I live by:

1. If you sell something online you should almost always use a landing page
2. If you using a landing page you should always split test multiple versions
3. If you’re not split testing right now – you need to start, immediately

What is Split Testing?

In affiliate marketing “Split Testing” a landing page is the process of comparing the performance of two landing pages to see if one converts better than the other. You compare them by “Splitting” your traffic between the pages.

For example: Your homepage has a yellow “Buy Now” button and has a click through rate (CTR) or 10% and a conversion rate (CVR) of 1%. You need to get your CTR to 15% in order to make a profit at 1% CVR. So – you split test to find something that will get you more clicks.

The Hypothesis

Each split test you do should be based on a hypothesis (a proposed explanation for a phenomenon). In this case – the hypothesis is that an orange buy now button could get more clicks and conversions than a yellow button.

Why Test Colors?

It has been well documented how much colors can impact the way people behave on your website. This isn’t theory this is a scientific fact.

For example: In a recent study, Scientists at the University of British Columbia studied more than 600 people. They found that the color red makes people more cautious and attentive to details while blue makes us more creative and open to new ideas.

Source: University of British Columbia, NPR.org, Science Daily, UVM.edu

So for this test this is our hypothesis:

Hypothesis: Changing the color of the Buy Now button will increase the number of clicks and conversion that our landing page gets.

The A/B Split Test

The most common method of split testing is the A/B method. To test the hypothesis (that orange might beat yellow) with the A/B method we split test two pages.

To create our test page we copy the existing page (Page A) and replace the yellow buy now button with the orange button and save it as a separate page (Page B) and upload it to the website.

Using CPVLab we split our web traffic into two groups, “A” and “B”. I also add the start date to the LP Name so I know when the test began.

One group is shown Page A and the other group is shown Page B. After running the test we can watch and see if Page B creates more clicks and conversions than Page A.

If it does – Page B wins the test and becomes the new landing page to beat and Page A gets thrown in the trash.

Benefits of A/B Split Testing

When you start a new campaign using a brand new landing page you will most likely lose some money. I’d say 90% of my campaigns lose money on day one.

The golden 10% that either break even or make money on their first day are a beautiful thing, but definitely do not happen every time.

So my only job is to transform the campaign from losing money to making a profit. I do this job by testing new things and improving performance.

This is the art of being an affiliate. It’s why some people can do it and some can’t.

“If you combine creativity with a relentless testing schedule – you can make almost any campaign profitable”. (Tweet This)

Things to Split Test

There are many things you can split test on a landing page. In fact – you can split test every element on a page if you have a lot of traffic. Here are a few of the things that I’ve seen make the most impact on page performance.

• Headlines
  • Probably the most impacting thing you change. Visitors read this first and decide whether or not to look at the rest of your page.
• Images
  • The first image a visitor sees can change the way they feel about your page.
  • A photo of the author helps build trust with the reader (if they like the photo)
• Call To Action buttons/links
  • Very important area to test. This is the text and button or link that goes to the offer, normally at the bottom of the page.
  • After reading your headline, looking at your image and scanning your copy – this is what they’re going to click (hopefully). Test it relentlessly.
• Copy
  • Obviously a big deal – this is what the user will read (or probably scan) and is your only chance to speak to them.
  • Learn to write killer copy with this book
    
• Captions under images
  • You’d be shocked at the difference captions (with links) can make on a page.
• Font Size
  • If your readers are age 40+ try increasing font size, as we get older it gets harder and harder to read tiny text on the screen
• Background colors
  • Colors are powerful. Red causes fear, Blue causes peace.

Downsides of A/B Split Testing

The problem with traditional A/B Testing is that if your new page (Page B) doesn’t perform as well as your original page (Page A) you will lose money on the 50% of your visitors that are shown Page B.

If Page A has a CTR of 10% and Page B has a CTR of 3% and your landing page gets 5000 visitors a day – you lose 350 click thru’s every day of your test.

Some tests require multiple days – or even weeks to get statistically significant results. So if Page B is performing poorly – you can lose a ton of cash.

Different Types of Split Testing

The type of split testing I do normally depends on the type of campaign I’m running and the value of each conversion – or cost per action (CPA).

To make decisions you need clicks and sales (data) and depending on the type of offer, getting statistically significant data can take a few minutes or a few days (or weeks).

For example: An auto insurance zip submit campaign (visitors enter their zip and click Submit) pays $2 per conversion brings in 1000s of conversions per day. I can make decisions pretty quickly. I get a nice, large pool of data to go by very fast.

A campaign for Rolex watches (visitors pay $2500 for a watch with a credit card) that pays $750 per conversion may only bring in 1 conversion per day. So I have to wait weeks to get the amount of data the insurance campaign gives me in 10 minutes.

Note: I recommend new affiliates start with low payout offers so you can get do tests like this without spending thousands of dollars and spending tons of time.

My Method: A/A/B Progressive Split Testing

No matter what type of offer you’re promoting and regardless of payout – A/A/B Progressive Split Testing is a method I suggest you try. It’s called A/A/B Progressive because we’re using 3 landing pages and the test builds slowly over time.

It takes a little longer to get results – but it also protects you from losing money on 50% of your visitors while your split test is running.

What is A/A/B Progressive Split Testing?

During a normal A/B split test you’d put Landing Page A up against Landing Page B, run 50% of your traffic to each page and see which one performs better.

During that test you are risking half of your traffic on the new page because it may or may not perform well.

To eliminate the risk – I give just 10% of my traffic to the new page. This way if it performs poorly, only a small percentage of my traffic is wasted.

“Why not just give Landing Page A 90% of the traffic and Landing Page B 10% of the traffic and see how things go?“

The Problem:

The 90% vs 10% A/B split test is flawed because the landing pages are getting different amounts of traffic at different times of day, which means the data could be inaccurate.

The Solution:

My goal is to give each page the exact same amount of traffic at the exact same time of the day for the exact same amount of time. This is the kind of test I can trust. But I don’t want to risk half my money by doing a 50/50 split.

How to Set UP A/A/B Progressive Split Test in CPVLab

Here’s what it looks like:
– Landing Page A (1) gets 80% of the traffic
– Landing Page A (2) gets 10% of the traffic
– Landing Page B gets 10% of the traffic

Benefits
The first benefit in this setup is that Landing Page B (the unproven test page) is only shown to 10% of my visitors. So if it performs horribly I’m only risking 10% of my profits (versus 50% in normal A/B tests).

The second benefit is that my split test Page A (2) versus Page B receives the exact same amount of traffic for the exact same period of time. This setup ensures that the data I get is as close to accurate as possible.

Potential Drawbacks

A drawback to this split testing style is that it takes a bit longer to get your data. When you launch an A/B test with each page getting 50% of the traffic you get to see how the page performs faster.

But losing money fast has never been my goal. I prefer to be patient and test slowly, because it is always more profitable in the long run.

Adjusting Your Split Test Over Time

Now that you have traffic coming in you’ll keep an eye on Page B and see how it does. If it appears that Landing Page B is going to do well (has great CTR or CVR) you will give the split test more traffic by raising it’s percentage in CPVLab to 20%.

To see if your page is going to do well – use this statistical significance calculator.

But remember – you want each page in the test to receive the exact same amount of traffic so when you increase page B to 20% you have to increase Page A (2) to 20% and decrease Page A (2) to 60% at the same time.

The new test looks like this:
– Landing Page A (1) 60%
– Landing Page A (2) 20%
– Landing Page B 20%

This way you can slowly increase the page, based on performance to speed up your test. Each page of the actual split test gets the same amount of traffic giving you more accurate data which the only important thing here.

*If Landing Page B is performing poorly, leave it at 10% even if it means your test takes more time. I’d rather test slowly than lose 50% of my revenue every day. If Page B never improves, pause it and create another page to test.

The Eventual A/B Split Test

If the new landing page seems to be performing as well or better than the original page you can eliminate Landing Page A (1) and move to a 50/50 split of the two pages.

You now know that Landing Page B is worthy of 50% of your traffic and will most likely not lose you a trunk full of money.

This looks like:
– Landing Page A (2) 50%
– Landing Page B 50%

That is the art of the A/A/B Progressive Split Testing. That is how you get to a 50/50 split test that won’t lose your money. This is the skill I wanted to teach you today.

A/A/B Progressive Split Testing on Prosper 202

If you are using Prosper202 and want to do this you will need to set up the pages manually in your index.php rotator file.

*I have not used Prosper202 for years and it may have been updated. But last time I used it, this is how I accomplished the goal.

To achieve an 80/10/10 split you’d need to add your Landing Page A (1) to the script 8 times and add Landing Page A (2) and Landing Page B to the rotator once each.

This looks like:
– Landing Page A (1)
– Landing Page A (1)
– Landing Page A (1)
– Landing Page A (1)
– Landing Page A (1)
– Landing Page A (1)
– Landing Page A (1)
– Landing Page A (2)
– Landing Page B

You would then do the progressive adjustments by deleting or adding page versions to the list. This is much more hassle and is one of the reasons I prefer paying for CPVLab.

Is A/A/B Progressive Split Testing Right For You?

In short – yes. I recommend this for every split test you do. I don’t see why anyone would do a normal A/B test unless they don’t mind losing some cash on the test, which is fine for super affiliates, but a bad idea if you don’t have unlimited funds.

No matter what style of landing page split testing you use I can’t stress enough the importance of split testing on a regular basis.

If A/A/B Progressive Split Testing isn’t for you – find something that works and do it all the time. Car salesman have the phrase “ABC” which stands for “Always Be Closing”. In Affiliate Marketing our phrase is “ABT” or “Always Be Testing”.

Split testing ads, landing pages and offers is the quickest way to take a campaign from loser to winner.

The Wrap Up

Today I’ve shown you a method of split testing that I use to improve my campaigns. It’s called A/A/B Progressive Split Testing because instead of testing two landing pages against each other (A/B) you test three and progress over time.

1. A (original at 80%)
2. A (original version 2 at 10%)
3. B (new page at 10%)

By using A/A/B Progressive Split Testing I can:

• Protect my profits while testing
• Get more accurate data during the test
• Eventually move to an A/B test that I know won’t lose money

I also gave you a solid list of things that I split test:

• Headlines
• Images
• Call to Action areas
• Copy
• Captions
• Font Sizes
• Background colors

If you’re not testing these things now – then get started. You could be leaving major amounts of money on the table.

 

But if you’re brand new and need a free system – Prosper202 can do the job. Just know that it is rarely updated and not supported.

Either way, get one and start split testing today.

Also: Get my free eBook ’10 Decisions’ about how I basically split tested my entire life

In Conclusion

The art of split testing is probably the most important skill an online marketer can attain. By mastering this skill – you’ll be able to take campaigns that lose money (90% of all campaigns) and turn them into a money makers.

Start with the headline and move down the list.

*Tip for new affiliates: If you have a smaller daily budget – don’t test more than one thing at a time or your test will take weeks to produce any real results.

Whether or not you choose to use A/A/B Progressive Split Testing as your testing method – you should Always Be Testing (ABT). Improving a campaign’s performance is your only job – so be good at it.

Question: Are you currently Split Testing? If so what method or system are you using?