Cold email A/B testing: Boost opens, clicks & replies with split tests

August 13, 2020

Reality check. It's extremely difficult to create the perfect cold email campaign on your first try.

People who managed to hit the bullseye that fast are rare. If you know someone with this skillset, let me know so we can become rich like Jeff Bezos. 😂

I'm not one of those people, so it takes a little trial-and-error and testing for me to figure it out. It's after that when I come up with the winning cold email.

There are no silver bullets, but there's A/B testing.

Cold email A/B testing is the ultimate way to quickly identify what works and what doesn't work in your outreach campaigns.

In this article, I'm going to show you all split test hacks and tricks I've learned that will help you ramp up your reply rate, boost the number of clicks, and get that open rate to go sky high.

Buckle up as we're about to go hard or go home.

What is A/B testing?

A/B testing is an experiment where you're split testing two or more variations at random, to figure out which variation brings better results.

In the world of cold emails, you can A/B test subject lines, two emails, different personalized images, or the entire sequences if you decide so.

cold email ab testing

How do you perform an A/B test with cold emails?

There are three important rules to live by:

  1. Test one thing at a time - e.g. test subject lines only, not both that and image vs. text
  2. Measure - always analyze all three cold email metrics (open, click, and reply rates) to get a clear picture what variation is working better
  3. Have a big-enough audience - to gain relevant insights, make sure your email list contains a minimum of 100-200 prospects.

What should you A/B test in your cold emails?  

A/B testing can have a significant impact on your outreach campaigns. It helps you accomplish goals faster in the short-term, as long as you know how to analyze your current cold email performance and identify existing problems.

Let's imagine a situation where you are trying to book meetings using cold emails.

To assess the current state of your outreach campaigns, we need to analyze the following:

  • Are people receiving and seeing your emails? Before analyzing any metric, make sure your email deliverability is clean
  • Are people opening your emails? Analyze your open rate
  • Are you using links or personalized video thumbnail cold email strategy? If so, the next step for prospects is to click on that link, so assess your click rates
  • How many replies that you get? Analyze your reply rate and type of replies you got

To make this even more actionable, I'll give you some example cold email stats. We're going to analyze them together.

bad cold email results to avoid

What problems do you see?

There are two KPIs that are on fire here, and not in a good way. People are not opening these emails. Plus, there are almost no replies.

Considering that the click rate is zero, let's assume there are no links in the cold email, and the sender's sole goal is to get more replies and book meetings.

KPI benchmarks to aim:

  • Open rate --> 50%+
  • Click rate –> 40%+
  • Reply rate --> 8%+

In our example, we know where our problems lie. Assuming that audience targeting is on point, people are not opening our emails and they're definitely not replying.

We need to fix that. Precisely where A/B testing can help.

Lemme show you the full power of cold email split tests.

A/B testing to increase open rates

Like I said before, before you start playing with open rates, make sure to check your email deliverability status.

When your open rate is lower than 50%, it might mean that people are not seeing your emails to begin with, so A/B testing subject lines would be a waste of time.

So you need to start with:

  1. Audit the current situation with your email domain
  2. Check your technical email settings (like SPF and DKIM)
  3. Start email domain warm-up

Once checked, you can move to A/B tests.

Finding the subject line

That's the first thing people see after receiving your cold email.

In other words, it makes a direct impact on your open rate.

Recommended read: Top 56 subject lines to test

Just click on the A/B test button. Add 2 different subject lines that you feel work better and don't change anything else.

lemlist will automatically split your audience in 2 groups and send emails accordingly.

how to ab test cold email campaign

Ideas to test when it comes to subject lines:

  1. Short vs. long subject lines
  2. Question vs. statement
  3. Add personalization to the subject line vs. general subject line

For example, after 5 different A/B tests, I found a subject line that gave me an 86% open rate. 😱

I started with a 55% open rate.


open rate ab testing

Experimenting with intro lines

That's the second thing people see before they open a cold email.

It should be catchy, so people will understand "it's not one more spammy email".

For example, here I played with the annoying "Hi {firstName}". Everyone knows that trick by now. So I got a little bit creative with it.

"Heyo {firstName}! Just joking Ted"

intro lines ab testing

Here are my results after A/B testing it with usual "Hi {firstName}:

open rate ab testing results
PRO TIP: You should think not only about the desktop version of your email but also mobile or even watches.

For example, this is the email that my friend Alex from Competera got. He uses Apple Watches to read emails.

how to write short cold emails

Send emails at different time

Although it might be a tricky thing to test and sometimes it's a waste of time, there are situations where it's a meaningful factor.

If you spot that your prospects are more likely to read emails at a certain point, due to being busy or having meetings, you can optimize for that.

In order to figure out if time is a factor, A/B testing is a great tool to do it. Try to make 2 different campaigns and send them on different schedules.

  1. Morning (from 8 to 11 am)
  2. After lunch (from 2pm to 5pm)

Boost your cold email click rate

There are different cases when you add links to cold emails. You can send links to articles, use cases, forms, calendar availability, and so on...

Since the majority of people try to generate leads with cold emails, I'll show you examples with links to "book a call" tools, such as Calendly.

Test different anchors

People always check the link before they click on it. If it looks weird, they won't click on it.

Yes, we marketers did a bad job during the last 15 years, by sending tons of spammy links... 😕

So one of the ideas is to A/B test the way your link looks in the cold email. For example:

  • full link:
  • link with anchor:
book more sales meetings

After 5 tests of different anchors, the results weren't different significantly. But in all test link with anchor won on +3-5% to the click rate.

Dynamic landing pages vs. regular cold email

I'll show you a tactic that you really need to A/B test. It gave me 2x more leads generated from cold emails.

The trick? Dynamic landing pages.

For example, there are 2 ways to send a Calendly link to your leads:

  • Add it inside the email:
book more sales meetings
  • Add a dynamic landing page with a custom video + embedded Calendly on it:
generate leads with cold emails

So, my email looked like this:

Best cold email template example

And when people click on the video, they go to a dedicated landing page, where they will see a few more personalized touchpoints, along with the Calendly link to book a demo with me.

And here are the results I got:


The difference was huge! 🔥

Recommended read: How to land sales meetings like

If you want to try dynamic landing pages, use lemlist for free for 14 days and get unrestricted access to all the cool features. 🔥

A/B testing to get more replies

The reply rate is the key metric for the majority of people who send cold emails.

The response is the beginning of a conversation that can potentially lead to a win-win situation.

"Tiramisu" strategy vs. generic intro

This is a strategy that we talk a lot in lemlist family, the largest closed community for salespeople.

It's about writing genuine intro lines and adding custom sentences that change for each prospect.

Vuk, lemlist's Head of Growth, created this mean Tiramisu cold email strategy when he was inviting sales experts to join our webinars.

tiramisu cold email strategy

The results were amazing! No, scratch that, A-M-A-Z-I-N-G!

We had Rand Fishkin, Aaron Ross, Tim Soulo, along many others joining. And btw, Vuk scored a 92% reply rate with this strategy.

Personalized image vs. text-only cold emails

With lemlist, you can personalize not only text but also the visual part. For example, you can add personalized images to your cold emails.

lemlist will automatically change text, logos, website screenshots and other elements on the image.

Here is the example of a cold email with personalized images:

Jeff's Zoom-meeting cold email strategy... Jeff was our lemlister of the week:

sales follow-up cold email template

For instance, you can decide to test a cold email template with text-only content vs. the one with a personalized image.

In my opinion, there is no "wow" effect comparing to the case when your recipients open a cold email and see their own website screenshot. So that's a great idea to A/B test.

You can find the most successful cold email templates here.

cold email hub

Liquid syntax A/B test

You can add personalization based on the info you have with liquid syntax.

For example, have "Hi {{fistName}}" if the location is the USA, and "Bonjour
{{firstName}}" if location is France.

how to personalize cold emails

You can also test different call-to-actions based on location.

how to personalize cold emails

That's a great thing to A/B test in your cold emails. You can easily personalize small things based on some general info that you have.

If you want to try it out, here is a how-to video by Vuk. 👇

Bottom line

A/B testing is the key to make your cold email perform better.

Don't stop when you find a template that works. Always try to improve and leverage split tests in the process.

I shared with you the ways you can improve your open, click, and reply rates.

If you liked this article, please share it with your friends and colleagues. ❤️

What you should look at next
cold emailing
How to get e-commerce clients with cold outreach [Cold Email Teardown]
Selling to e-commerce companies has its perks, but it also requires breaking through the insane level of competition. Check out how Bill, CEO of Sales Captain, would target and convert e-commerce clients with one email template!
Lilibeth Acuna
September 28, 2023
cold emailing
How to beat email sending limits (and reach more prospects!)
Email sending limits can slow down your cold outreach - but if you ignore them, you might get flagged as spam. Here’s how to reach more leads without hurting your email deliverability!
Julia Hamilton
September 27, 2023
cold emailing
How to sell web and app development services [Cold Email Teardown]
Outreaching prospects to sell web and app development services without success? Let Mathieu, the Head of Email Marketing at Cactus Marketing, show you his tips and tricks for boosting your reply rates!
Lilibeth Acuna
September 21, 2023
cold emailing
20+ email greeting examples to use in your next cold email
Check out 20+ email greeting examples to help you start the email on the right note and book more meetings.
Julia Hamilton
September 13, 2023
cold emailing
How to personalize your outreach at scale with advanced conditions [2023 guide]
Discover the key to best growth teams’ outreach success - advanced conditions. Get 5 steps on how to add them to your campaign & engage with leads at scale, in a human way.
Mihaela Cicvaric
September 13, 2023
cold emailing
Top 15 multi-channel campaigns with advanced conditions for higher conversions
Scale your outreach with 15 multi-channel campaigns with advanced conditions triggered by your leads' actions. 4X your conversions in a human way.
Mihaela Cicvaric
September 7, 2023

Book your
first meetings
this week!

Accelerate your business growth and start generating revenue with lemlist
Try for free
No card required.
Try for free
No card required.