Cold email A/B testing: Get more opens, clicks & replies with split tests

It's quite difficult to create the perfect cold email campaign on your first try.

There are no silver bullets, but there's A/B testing.

Cold email A/B testing is the ultimate way to quickly identify what works and what doesn't work in your outreach campaigns.

What is A/B testing?

A/B testing is an experiment where you're split-testing two or more variations randomly, to figure out which variation brings better results.

In the world of cold emails, you can A/B test subject lines, two emails, different personalized images, or the entire sequences if you decide so.

How do you perform an A/B test with cold emails?

There are three important rules to live by:

1. Test one thing at a time: e.g. test subject lines only, not both that and image vs. text
2. Measure: always analyze all three cold email metrics (open, click, and reply rates) to get a clear picture of what variation is working better
3. Have a big enough audience: to gain relevant insights, make sure your email list contains a minimum of 100-200 prospects.

What should you A/B test in your cold emails?

A/B testing helps you accomplish goals faster in the short-term, as long as you know how to analyze your current cold email performance and identify existing problems.

Let's imagine a situation where you are trying to book meetings using cold emails.

To assess the current state of your outreach campaigns, we need to analyze the following:

• Are people receiving and seeing your emails? Before analyzing any metric, make sure your email deliverability is clean
• Are people opening your emails? Analyze your open rate
• Are you using links or personalized video thumbnail cold email strategies? If so, the next step for prospects is to click on that link, to assess your click rates
• How many replies did you get? Analyze your reply rate and the type of replies you got

To make this even more actionable, let’s analyze some example cold email stats.

What problems do you see?

Two KPIs are standing out here, and not in a good way.

People are not opening these emails. Plus, there are almost no replies.

Considering that the click rate is zero, let's assume there are no links in the cold email, and the sender's sole goal is to get more replies and book meetings.

Goal KPI benchmarks:

• Open rate → 50%+
• Click rate → 40%+
• Reply rate → 8%+

In our example, we know where our problems lie.

Assuming that audience targeting is on point, people are not opening our emails and they're definitely not replying.

We need to fix that. Precisely where A/B testing can help.

A/B testing to increase open rates

Before you start playing with open rates, make sure to check your email deliverability status.

When your open rate is lower than 50%, it might mean that people are not seeing your emails to begin with, so A/B testing subject lines would be a waste of time.

So you need to start with:

1. Audit the current situation with your email domain
2. Check your technical email settings (like SPF and DKIM)
3. Start email domain warm-up

Once checked, you can move to A/B tests.

Finding the subject line

That's the first thing people see after receiving your cold email.

In other words, it makes a direct impact on your open rate.

Recommended read:
Top 56 subject lines to test

Just click on the A/B test button.

Add 2 different subject lines that you feel work better and don't change anything else.

lemlist will automatically split your audience into 2 groups and send emails accordingly.

Ideas to test when it comes to subject lines:

1. Short vs. long subject lines
2. Question vs. statement
3. Add personalization to the subject line vs. the general subject line

For example, after 5 different A/B tests, I found a subject line that gave me an 86% open rate. 😱

I started with a 55% open rate.

Results:

Experimenting with intro lines

That's the second thing people see before they open a cold email.

It should be catchy, so people will understand "It's not just another spammy email".

In this example, I tested "Hi {firstName}" against "Heyo {firstName}! Just joking Ted".

Here are my results after A/B testing it with usual "Hi {firstName}:

PRO TIP: You shouldn’t only consider the desktop version of your email, as most people receive their emails in their phones or smart watches.

Send emails at different times

Although it might be tricky to test and sometimes it's a waste of time, there are situations where it's a meaningful factor.

In order to figure out if time is a factor, A/B testing is a great tool to do it.

Try to make 2 different campaigns and send them on different schedules.

1. Morning (from 8 to 11 am)
2. After lunch (from 2pm to 5pm)

Also, consider the country the person you are targeting is in, as timeframes may vary.

Boost your cold email click rate

There are different cases when you add links to cold emails.

You can send links to articles, use cases, forms, calendar availability, and so on...

Since most people try to generate leads with cold emails, I'll show you examples with links to "book a call" tools, such as lemcal.

Test different anchors

People always check the link before they click on it.

If it looks suspicious, they won't click on it in case it’s spam.

So one of the ideas is to A/B test the way your link looks in the cold email. For example:

• full link:
• link with anchor:

After 5 tests of different anchors, the results weren't different significantly.

But in all tests, links with anchor won on +3-5% to the click rate.

Dynamic landing pages vs. regular cold email

You really need to A/B test this tactic, it gave me 2x more leads generated from cold emails.

The trick? Dynamic landing pages.

For example, there are 2 ways to send a lemcal link to your leads:

• Include it in the email:

• Add a dynamic landing page with a custom video + embedded lemcal on it:

So, my email looked like this:

When people click on the video, they go to a dedicated landing page, where they will see a few more personalized touchpoints, along with the lemcal link to book a demo with me.

And here are the results I got:

The difference was huge! 🔥

Recommended read:
‍How to land more sales meetings

If you want to try dynamic landing pages, use lemlist for free for 14 days and get unrestricted access to all the cool features. 🔥

A/B testing to get more replies

The reply rate is the key metric for the majority of people who send cold emails.

The response is the beginning of a conversation that can potentially lead to a win-win situation.

"Tiramisu" strategy vs. generic intro

This is a strategy that we talk a lot about in lemlist family, the largest closed community for salespeople.

The Tiramisu strategy consists of writing genuine intro lines and adding custom sentences that change for each prospect.

The results were amazing!

This strategy scored a 92% reply rate.

Personalized image vs. text-only cold emails

With lemlist, you can personalize not only the text but also the visual part.

For example, you can add personalized images to your cold emails.

lemlist will automatically change text, logos, website screenshots, and other elements on the image.

Here is an example of a cold email with personalized images:

For instance, you can decide to test a cold email template with text-only content vs. the one with a personalized image.

You can find the most successful cold email templates here.

Liquid syntax A/B test

You can add personalization based on the info you have with liquid syntax.

For example, have "Hi {{fistName}}" if the location is the USA, and "Bonjour

{{firstName}}" if the location is France.

You can also test different call-to-actions based on location.

That's a great thing to A/B test in your cold emails.

You can easily personalize small things based on some general info that you have.

Bottom line

A/B testing is the key to making your cold email perform better.

Don't stop when you find a template that works. Always try to improve and leverage split tests in the process.

Try your 14-day free trial at lemlist now and start A/B testing your campaigns!