How many of your cold emails actually get opened? Half, if you’re lucky? Most reps quietly accept a mediocre open rate as the price of doing outbound. I did too, for a long time, until I started testing instead of guessing. Then I watched my open rate climb from 55% to 86% off nothing but subject-line experiments.

In this post, I’ll show you how to run cold email A/B tests that actually move your numbers. I’ll cover the manual split tests you already know, on subject lines and copy, plus a newer layer most guides skip: testing the AI that now writes, personalizes, and times your outreach.

A/B testing is an experiment where you’re split-testing two or more variations randomly, to figure out which variation brings better results.

In the world of cold emails, you can A/B test subject lines, two emails, personalized images in cold emails, or the entire sequences if you decide so.

There are three important rules to live by:

A/B testing helps you accomplish goals faster in the short-term, as long as you know how to analyze your current cold email performance and identify existing problems.

Let’s imagine a situation where you are trying to book meetings using cold emails.

To assess the current state of your outreach campaigns, we need to analyze the following:

To make this even more actionable, let’s analyze some example cold email stats.

What problems do you see?

Two KPIs are standing out here, and not in a good way.

People are not opening these emails. Plus, there are almost no replies.

Considering that the click rate is zero, let’s assume there are no links in the cold email, and the sender’s sole goal is to get more replies and book meetings.

Goal KPI benchmarks:

In our example, we know where our problems lie.

Assuming that audience targeting is on point, people are not opening our emails and they’re definitely not replying.

We need to fix that. Precisely where A/B testing can help.

Before you start playing with open rates, make sure to check your email deliverability status.

When your open rate is lower than 50%, it might mean that people are not seeing your emails to begin with, so A/B testing subject lines would be a waste of time.

So you need to start with:

Once checked, you can move to A/B tests.

That’s the first thing people see after receiving your cold email.

In other words, it makes a direct impact on your open rate.

Recommended read: Top 56 subject lines to test

Just click on the A/B test button.

Add 2 different subject lines that you feel work better and don’t change anything else.

lemlist will automatically split your audience into 2 groups and send emails accordingly.

Ideas to test when it comes to subject lines:

For example, after 5 different A/B tests, I found a subject line that gave me an 86% open rate.

I started with a 55% open rate.

Results:

That’s the second thing people see before they open a cold email.

It should be catchy, so people will understand “It’s not just another spammy email”.

In this example, I tested “Hi {firstName}” against “Heyo {firstName}! Just joking Ted”.

Here are my results after A/B testing it with usual “Hi {firstName}”:

PRO TIP: You shouldn’t only consider the desktop version of your email, as most people receive their emails on their phones or smart watches.

Although it might be tricky to test and sometimes it’s a waste of time, there are situations where it’s a meaningful factor.

In order to figure out if time is a factor, A/B testing is a great tool to do it.

Try to make 2 different campaigns and send them on different schedules.

Also, consider the country the person you are targeting is in, as timeframes may vary.

Better still, stop guessing entirely. lemlist’s intent signal agents watch for buying moments like website visits, funding, and job changes, so your test becomes “which message wins” rather than “which random hour wins”.

There are different cases when you add links to cold emails.

You can send links to articles, use cases, forms, calendar availability, and so on…

Since most people try to generate leads with cold emails, I’ll show you examples with links to “book a call” tools, such as lemcal.

People always check the link before they click on it.

If it looks suspicious, they won’t click on it in case it’s spam.

So one of the ideas is to A/B test the way your link looks in the cold email. For example:

After 5 tests of different anchors, the results weren’t significantly different.

But in all tests, links with anchor won on +3-5% to the click rate.

You really need to A/B test this tactic, it gave me 2x more leads generated from cold emails.

The trick? A dynamic landing page, one of the tactics I break down in this guide to personalize your outreach at scale.

For example, there are 2 ways to send a lemcal link to your leads:

So, my email looked like this:

When people click on the video, they go to a dedicated landing page, where they will see a few more personalized touchpoints, along with the lemcal link to book a demo with me.

And here are the results I got:

The difference was huge!

Recommended read: How to land more sales meetings

If you want to try dynamic landing pages, use lemlist for free for 14 days and get unrestricted access to all the cool features.

The reply rate is the key metric for the majority of people who send cold emails.

The response is the beginning of a conversation that can potentially lead to a win-win situation.

The Tiramisu strategy consists of writing genuine intro lines and adding custom sentences that change for each prospect.

The results were amazing!

This strategy scored a 92% reply rate.

With lemlist, you can personalize not only the text but also the visual part.

For example, you can add personalized images to your cold emails.

lemlist will automatically change text, logos, website screenshots, and other elements on the image.

Here is an example of a cold email with personalized images:

For instance, you can decide to test a cold email template with text-only content vs. the one with a personalized image.

You can find the most successful cold email templates here.

You can add personalization based on the info you have with liquid syntax.

For example, have “Hi {{firstName}}” if the location is the USA, and “Bonjour {{firstName}}” if the location is France.

You can also test different call-to-actions based on location.

That’s a great thing to A/B test in your cold emails.

You can easily personalize small things based on some general info that you have.

Most A/B testing guides stop at subject lines and images you swap by hand. That still matters. But the bigger wins now come from testing the AI layer itself.

Start with full sequences. Build a variant with lemlist’s AI campaign builder, lemAgent, and run it head-to-head against the sequence you wrote yourself. Then test personalization. Put AI-personalized variables and voice notes up against plain merge-tag copy and see which one earns replies. Do the same with research: pit personalization from AI enrichment agents against your static templates. And stop guessing at send times. Let intent-based timing decide when a prospect actually hears from you.

Here’s the honest part. AI copy can read generic, and prospects can smell a template from a mile away. So the AI still needs real context to win, the actual research and the reason you’re reaching out right now. The upside is worth it. Gartner found that sellers who partner with AI are 3.7 times more likely to meet quota, and Salesforce reported that 83% of sales teams using AI saw revenue growth versus 66% without.

A winner isn’t a winner until the math says so. How big your test needs to be depends on two things: your baseline reply rate and the size of the lift you’re trying to catch. A jump from 5% to 15% shows up fast. Proving a 1% difference takes far more sends. So there’s no single magic number of emails that makes a test valid, and anyone who hands you one is guessing.

What I do instead: I wait for a meaningful confidence level, around 95%, before I call a winner. And I keep a floor of at least 100 to 200 prospects per variation, so a few lucky opens don’t fool me. Bigger, cleaner lists make every result more trustworthy. Junk data just gives you confident nonsense.

A few traps I see reps fall into over and over.

Testing two things at once. If you change the subject line and the CTA in the same test, you’ll never know which one moved the number.

Calling it too early. That exciting day-one lead often disappears by day three.

Running it on a tiny list. Ten opens out of twenty proves nothing.

Judging on opens alone. A subject line can win the open and still tank your replies, and replies are what actually pay you.

A/B testing is how you stop guessing and start compounding small wins into a real edge. Test your subject lines and copy by hand, then test the AI layer that now writes and personalizes your sequences. Keep the variants that win. Kill the ones that don’t. That loop is what took my open rate from 55% to 86%, and it’s what keeps improving long after that. Want to run your first test this week? Start a 14-day free trial and put two variants live today.