If you use email marketing, you’ve probably spent longer than you care to admit debating whether or not to include an emoji in your subject line or whether the ‘Order Now’ button should be red or green. You know these small factors can make a big difference in the open, click-through and purchase rates of your emails… if only there were a way to look into the future and predict which tweaks your audience would respond to best.
Enter: multivariate testing.
Multivariate testing is an email marketing feature that allows you to predict which of a series of changes will lead to the best results and tailor your campaigns accordingly. Here, we’ll explain why multivariate testing is so beneficial in email marketing and how you can use it to drive more engagements and greater revenue from every email you send.
What is Multivariate Testing?
You’ve probably heard of A/B testing, a conversion rate optimization tactic in which two similar versions of a marketing item like a landing page or email are tested against one another. A/B testing generally refers to just two versions of the item being tested. In multivariate testing, you can test many different variations on the same content piece.
The great thing about multivariate testing for email is that you establish the results quickly, before your entire audience receives the campaign in question. This way, you can ensure as many subscribers as possible are sent the “winning” campaign.
Here’s how it works. First, you set up your campaigns. Each of them will have slightly different variations to things like the subject line or pre-header text. Once your variables have been established, you’ll set a test duration and a sample size. This is the “test” population.
During the test duration, i.e. one hour, your sample audience will receive the different variations of your email in equal proportions. So, if you had a sample size of 600 and three options to test, each of the options would be sent to 200 subscribers.
After the predetermined time frame is up, the results are analyzed to determine which version of the email is the winner. You can base the “winner” on a number of metrics, including open rate, number of clicks, or resulting revenue, depending on what’s most important for that campaign. Once you’ve established a winner, that email will be sent to the remainder of your audience, maximizing the campaign’s likelihood of success.
What Should I Test With Multivariate Testing?
Subject line and preview text. Test out different lengths, topics, word choices, use of emojis, and use of the customer’s name.
‘From’ name. Try different variations of the company’s name, like ‘Acme’ versus ‘Acme Furniture Company,’ or use an individual’s name, like ‘John at Acme Furniture.’
Email design. Long or short? One column or two? Image heavy or text only? Test header sizes, font sizes, logos, image positions, and image variations.
Colors. This might include the color of your headers, accents, background or calls to action.
Call to action. Try changing the phrasing, format (button versus text), size, and CTA placement within the email.
4 Best Practices For Boosting Email Performance With Multivariate Testing
Use an Established List
In order for multivariate testing to be effective, we recommend using it on lists of 5,000 subscribers or more. This ensures you not only have a large enough pool from which to draw an adequate sample, but that your results aren’t skewed by your list being too new. If the majority of people you’re emailing aren’t used to hearing from you on a regular basis, this could result in higher-than-normal unsubscribe rates.
Have a list that’s smaller than 5,000 subscribers? No worries! You can still take advantage of conversion rate optimization. In this case, we’d recommend doing a simple A/B test where half of your subscribers are sent one variation of a message and half are sent another. The results can be analyzed and used to make decisions in future campaigns.
Use a Reliable Sample
This is another reason the list you start with needs to be of adequate size. To get results that are considered statistically significant, we recommend a sample audience of at least 500 test subjects.
Within any sample for any type of test, there’s going to be random variation (i.e. results that are based purely on chance rather than being indicative of a meaningful trend). The smaller the size of your sample, the less reliable it is. For most lists, a sample audience of 10 to 20% is a good benchmark.
Test One Variable At A Time
While multivariate testing allows you to test as many variables as you want in any given test, your best best is to test just one item at a time (i.e. subject line or email design, but not both). The reason being that if you test multiple variables, there’s no clear way to know which of them contributed to the winning results, which is important intel if you want to continuously improve your email performance over time.
Additionally, the more options you test, the bigger sample size you’ll need, as each additional option splits the target audience into smaller test groups.
Examine Results Over Time
Ideally, multivariate testing should be just one tactic in your overall email optimization strategy. There are numerous factors that can influence the results of a multivariate test, from the day of the week to the time of day of your send, holidays and even external factors like major weather events. Holiday season in particular is a tricky time to make broad assessments based on multivariate tests because subscribers’ email behavior is different than normal.
So, don’t just run one test and think of it as gospel when it comes to future subject lines/email layouts/etc. Instead, conduct tests on an ongoing basis and use the results to expand your ever-growing pool of knowledge about what resonates best with your subscribers.
Supercharge Your Email Performance with Multivariate Testing from Springbot
Springbot is pleased to announce the rollout of multivariate testing, the latest tool in our user-friendly tool kit to help marketers get the most out of every campaign. Just a few clicks is all it takes to set up an audience sample and determine your testing variables.
Learn more about how to set up your first multivariate test here. New to Springbot? Contact us today to schedule your free demo and learn how to harness the power of automation and personalization for more profitable marketing on every channel.
