Home >Blog >How A/B Split Testing Improves Digital Marketing Strategies
A/B split testing

How A/B Split Testing Improves Digital Marketing Strategies

Nicki Betterbid

There was a time when measuring the effectiveness of multiple marketing campaigns was problematic at best, and at worst, mere guesswork. Companies rarely, if ever, knew exactly which campaign drove the customer to their door. However, with today’s digital marketing strategies, companies can analyze results at a granular level and one of the tools they can use to improve results is A/B split testing.

What is A/B Split Testing?

So, what exactly is A/B split testing and how can it be used to improve your digital marketing campaigns? Simply put, A/B split testing involves measuring the results between two or more digital marketing campaigns where small individual changes are analyzed to see what impact they have on the actions customers take.

A/B split testing can be used to improve multiple digital marketing approaches. You can analyze the impact of small changes and adjustments on a given landing page, to a website’s design, to an email campaign, or Pay-Per-Click (PPC) advertising campaigns.

The key is to establish baseline results with a control group. The control group essentially defines how your campaign initially performed. Each subsequent campaign includes small changes where the results are then compared to the original baseline results.

Digital campaigns can be run simultaneously to provide a more comprehensive representation of how different changes impact results. However, the key is to make no more than one change at a time for each campaign. Making more than one change per campaign will not allow you to pinpoint which change made the all-important difference.

Running an A/B Split Test on an Email Campaign

We’ll analyze how an A/B split test would work with an email marketing campaign. We will define our initial baseline results and then make one change at a time and measure the success or failure of that change. Let’s assume we’ve been running the same email campaign layout for several months.

We send this one email campaign to 100,000 recipients and receive an average of 800 customer actions, which means we have a click-through rate of 0.80%. We will consider this our baseline email campaign and measure all future changes against this baseline.

AB Testing Email Marketing

Month 2 Changes

In the second month, we made one change only. We decided to combine the “Attention-Grabbing Title” and “Sales Pitch / Offer” into one cohesive eye-catching headline that’s now positioned right under the company logo. The result of this single change means that we now get 1000 customer actions out of 100,000 recipients, which means our CTR is now 1.00%. This is an increase from our baseline email campaign. 

Month 3 Changes

We keep the change we made from the second month and make one more change in the third month. This time, we move the “Content: About the Offer” immediately below our new heading, which then moves the three separate videos below the core content. This change now means we get 1310 customer actions out of 100,000 recipients, which leads to a CTR of 1.31%.

Month 4 Changes

Again, we keep the same changes from the previous 2 months, but now we have completely removed the company advertisements on the right-hand side of the email. We do this because we feel it distracts from our messaging. This one change now sees us achieve 1400 customer actions out of 100,000 recipients, which gives us our best CTR to date at 1.4%.

Month 5 Changes

This month we decided to remove one of the “Overstock Items for Sale” icons on the left-hand side. Unfortunately, only 1320 customer actions occurred out of 100,000 recipients, which means our CTR has now declined to 1.32%. While this may not seem like much of a decline, the company will easily be able to quantify the decline by lost revenue, lost opportunities, or fewer customer signups.

AB Split Testing Analysis Email Campaign

Understanding Variables to Change

Multiple variables can affect your email campaign’s success. We’ve simplified this process by combining headlines and moving sections around, which do play a critical role in improving results. However, other changes can have a more profound effect. Here are some variables to consider.

  1. Fonts: Yes, font styles and sizes can have a direct effect on digital marketing campaigns. This is especially the case with PPC campaigns. Serif fonts are ideal when marketing high-end, sophisticated products, which is why they’re the preferred fonts for jewelry and clothing brands. Display fonts are great for eye-catching headlines and when looking to make your brand stand apart from the crowd. Each font serves its own purpose, so take the time to investigate which font is best for your message. 
  2. Colors: Colors are another important variable. The colors of your fonts can play a critical role in the success of every digital campaign. There’s a reason why Google Ads performs best in blue, as it conveys competence, reliability, and authority. Red conveys excitement, daring, and imagination. Brown conveys ruggedness and strength, which is why hunting and camping brands rely upon it so often.
  3. Images: You can’t just rely upon the written word when looking to engage your audience. Ultimately, images are essential in any digital marketing strategy. However, not just any image will do. High-definition images with vibrant colors immediately attract people’s attention, while nondescript and low-quality images are immediately glossed over and ignored. Invest in generating high-quality images. It makes a difference.
  4. Maximizing Whitespace: One of the biggest mistakes marketers make with email campaigns, website design, and their landing pages, is to not maximize white space. PPC campaigns are effective because there’s no clutter. A PPC campaign delivers a clear and concise message to your audience. However, it’s common for people to occupy whitespace on websites and landing pages with excessive videos, images, links, etc. Unfortunately, all this clutter only confuses your audience and takes away from your core message.
  5. Design and Layout: Our example showcased the impact that design and layout can have on an email campaign. However, it plays an equally important role in website design and how users interact with a given landing page. Your message and offer should be above the fold, which is the upper half of your email campaign or landing page. This is where you state your message, present your offer and position your best argument. The below the fold portion should include ancillary support information such as the terms and conditions of the offer and your contact details.

AB split testing is a relatively straightforward process. However, it’s not a one-time event. It’s about continuous improvement. Always review the results of your PPC and email campaigns and be willing to invest time in analyzing the results of any changes you decide to make. 

At PBJ Marketing, we have invaluable experience running multiple different types of digital marketing campaigns. We are well-versed in the ins and outs of measuring performance and improving upon results. If you are looking for a partner to handle your next email or PPC campaign, then contact us now.