A/B Testing And Your Email Newsletter

The A/B testing theme continues!

So far, we’ve talked about A/B testing in a very general way. Then, we specifically discussed A/B testing related to websites where I listed at least twenty specific (i.e. annoyingly small) things you could test. Then, we talked about using A/B testing ideas in a social media setting (using some of the principles to test elements like time of day for posting). The final frontier is to talk about email marketing and A/B testing, which is what this post is all about.

Now I will be the first to say it seems DUMB that sending something at a slightly different time or having slightly different wording could have any affect on anything. Really. If you are reading this series of blog posts shaking your head, I am right there with you as I write them. Changing one small thing shouldn’t matter, but it turns out…it can. So we can either ignore evidence and leave money on the table, or shrug and consider the possibility.

I warned our email list that we were going to experiment on them for this blog post. We typically send out our newsletter on or around the 19th of every month, whenever the closest Friday is. So we sent our monthly email newsletter (about… A/B testing. I know, shocker.)

Mailchimp has a way you can send A/B campaigns. It’s the part I tell people to ignore when I am first training them, but when they come back for round two of training (or even just a longer format Intro To Email Marketing class), I always touch on the concept.

To keep things simple, we only experimented with timing. I sent half our list the email at 1:00 PM Friday and the rest at 6:15 PM the same day. I mean, how much different could it be, right?

So by simply sending the email later on Friday, doubled the open rate and more than doubled the click through rate. 

What else can you test with your email besides what time you send it?

• Date you send email
• Subject line text
• Subject line with emoji (you can test emojis or subject line with and without)
• Button color in newsletter (pretty much everything visual/layout wise like you can in websites)
• Length of newsletter
• Newsletter with and without embedded video (overall formatting the same information in different ways)
• Lots more.

The point of all these posts? Simple:

1. Stop thinking you know what your customers are doing and actually test it.
2. In a lot of cases, it’s not difficult to do A/B testing, you just need to decide what you want (that’s the hard part).
3. It really does seem to make a difference. Imagine how much little changes could add up over time.

Feel free to check out all our A/B Testing blog entries and get fulling informed on this very cool (and actually effective) idea.