Customer Blog Takeover: You’re Testing That Wrong if…

Posted by Mandy Parisi 

Working as an account manager at a major email service provider was a wonderful opportunity. I spent 5 years learning the ins and outs of a variety of programs – retail, ecommerce, B2B and even a video store. Yes, I’m dating myself here. Kids, that used to be a thing! You’d walk around and look at new releases and then….oh never mind.  

Of course, being an account manager wasn’t all sunshine and puppy dogs. Anyone who’s been in a similar role will tell you that some clients are better than others.

I’ve had clients who loved to wait until 5 pm on a Friday before they decided to change the creative on an offer going out Saturday morning. Certain clients liked to use very colorful magic words in the rare case of a QA miss. I didn’t really mind a late night or inventive use of English language – it’s all par for the course. What really ground my gears was this: a complete lack of strategy around testing.

Whether you’re brand new to the email marketing world or a seasoned wiz, the directive coming from leadership is usually Test, Test, Test. Sure, it’s easy to slap together a subject line A/B or pit one template against another. But, it can be really time-consuming. And, what I found too often was that clients didn’t leverage any of the findings.

Having a defined testing strategy is just as important as impactful creative, targeting users with the right message at the right time, and maintaining data quality. By the way, if you need help with the latter, I know this really cool company with a great Certification program.

Companies large and small are guilty of willy-nilly testing strategies. Let’s take for example a very well-known e-commerce retailer who shall remain unnamed. Three or 4 times a week, we’d run some kind of 50/50 subject line test. One day, it would be a discount offer vs free shipping. The next day, it would be a green shirt vs a blue shirt. Going into the weekend, let’s go with header navigation vs no navigation. It was testing for testing’s sake. It’s testing so you can tell your CMO you’re testing.

To be fair, maybe I didn’t have full visibility into my ex-client’s overall strategy. Maybe they were stashing the results away in some kind of master marketing tomb to create the ultimate customer profile. It sure didn’t feel that way.

In any case, there are some very basic rules for differentiating a good test from a bad test, whether you’re about to send your first email or your 35,000,000th.

Here are three tips to keep in mind:

  • Set specific KPIs – What is the goal of your test? Are you trying to increase your average order value? Grow your list? Understand how to make better product recommendations? If you don’t know what your key performance indicators are, don’t bother testing. The idea of testing is that you use the results to make your program better. Figure out what needs to change first. 
  • Make results actionable – If your creative department is 6 months behind and you know there’s no way you can change your email template…don’t test your email template.  Partner with other teams in your organization, like IT and web production, to understand what’s an easy win and what’s not going to happen this quarter…or year. 
  • Go for quick wins – Unless you’re using the results quickly, what works this week may not work the next. That’s why I’m a big fan of the 10/10/80 test. Ten percent of the audience gets one experience, 10% gets another experience, and then after a set amount of time (commonly 24 hours), the remaining 80% gets the winning experience. Boom. You’re capitalizing on how your audience is feeling at the moment. Salesforce Marketing Cloud is one email service provider with an automated A/B testing tool that makes the setup on this really easy.

I know it can be hard to pull out of the day-to-day grind to look at strategy from a higher level, but it’s well worth the exercise. Better tests mean better results. Better results mean satisfied CMOs. Satisfied CMOs mean happier email marketers.

Author Image

About Mandy Parisi

Mandy is the senior manager of onboarding and retention as Webroot, an endpoint security and threat intelligence services company dedicated to protecting individuals and businesses from online threats. Before moving client-side, she managed major accounts including U.S. Bank and Hotwire at Yes Lifecycle Marketing in Portland, OR. In her free time, she enjoys the Colorado sunshine with her husband and two little dogs.

Author Archive