Straight Up vs Sexy: Testing Email Subject Lines
By Carly Stec
If you ask a marketer what goes into a great email subject line, it's likely that you'll get a dozen different answers.
If you Google it, you'll be met with several dozen more.
With so much riding on the performance of an email's subject line, we took matters into our own hands and conducted an A/B test comparing two different subject lines promoting the same offer.
The results are in and we're ready to walk you through it. Let's go.
Here's what the A/B test looked like:
Variation A: The Newly-Updated Beginner's Guide to Inbound Marketing
Variation B: Attract and convert more customers on your website right now
Again, same email, same offer, different subject line.
While variation A explicitly states what the recipient is receiving, variation B works to convey the value of the guide without making mention of it.
Here's what we found:
Right off the bat we noticed that the test had little to no effect on whether or the recipients decided to open the email.
At first we couldn't believe it.
Seriously? A .5% discrepancy? Is this real life?
Once the initial shock wore off, we revisited the data and redirected our attention to click-to-open rate.
While 10.9% of the people who opened variation A clicked-through to the offer, only 7.8% of the people who opened variation B clicked.
Aha! It appeared that we were on to something, but what?
While the mismatched subject lines had little influence on a recipient's decision to open the email, it did, in fact, impact their decision to click-through.
Here's what you should know:
According to MailChimp, “the best subject lines tell what's inside, and the worst subject lines sell what's inside.”
Our test results revealed that people are more likely to engage with the content inside your email if you clearly set the expectation in the subject line.
The recipients of variation A knew what they were getting into before they opened the email. The subject line was straight up.
When they clicked into the email, The Beginner's Guide was there waiting for them, as expected. They clicked.
The recipients of variation B received something a bit sexier. More actionable. More benefit-driven. Less explicit.
When they clicked into the email, they were met with an offer that they may or may not have been expecting given the ambiguity of the subject line. They clicked less.
This doesn't mean that you should flee to your email composer of choice and start blasting out definitive subject lines. Let's not get carried away.
But it does serve as valuable data to call up when deciding between one subject line or another.
Again, this is not the end all be all.
Like any good A/B test, we'll repeat this experiment several times in an effort to replicate the results and ensure validity.
We encourage you to do the same. The more data-backed email subject line best practices out there the better.
Wondering where to begin?