Enhancements for Email Betting, I Mean Testing

We love email testing at Knotice. When it comes to our own campaigns, we like to up the ante - literally. For each test, there is usually a corresponding betting pool going on. Granted, testing provides great marketing insights; but, the drama increases when people start wagering on how smart of marketers they are. The only exception is Bryce. He actually bets against what he thinks will be the top performer, and so far he's in the lead. Hey, whatever works for him.

One of our infamous tests was when we tested the traditional “rich” format that companies use all the time for their B2B email marketing against a “simple” format. This was for a webinar we were promoting. Both types came from our sales reps with the same subject line, same content, etc. The results were pretty astounding…

Pretty amazing, huh? A 500% increase in the click-to-open rate. The 100% increase in conversions wasn’t too shabby either. Just think how many marketers simply roll with the rich format because… well… they do. They can’t give any quantifiable reason why they do, they just do. There are so many untested assumptions, conventions, and opinions that still rule interactive marketing. It blows me away.

Part of my job here at Knotice is to make testing and optimization simple. Testing has been around for awhile. The problem is, though, it’s usually not the easiest thing to do. In part, that’s why there are so many untested conventions and opinions still floating around interactive marketing departments.

In the latest release of Concentri, we rolled out some refinements to our email a/b testing interface. As you’ll see, the refinements make things pretty simple.


You can create a new test just like you’d create any other campaign element in Concentri.


We changed the names of test types so they are explicit (i.e. 1 email with multiple subject lines).


We made it really easy to add multiple subject lines for a subject line test. Oops, I mean a “1 email with multiple subject lines” test.


This was a big one. We made test scheduling work just like if you were to schedule an email, sms text campaign, or anything else in Concentri.


Here’s the money screen. At a quick glance you can see the state of the test after it’s been scheduled, and the current test results. The results are also available under the “Tests” section of our reporting environment.

Next up, we’re already focusing on making multivariate testing easy. Stay tuned for that and other enhancements. How does a “bookie feature” for taking wagers on tests in Concentri sound? 🙂

2 Comments

  1. Jeffry
    Posted December 21, 2008 at 11:09 am | Permalink

    If you can identify the targets who have the highest correlation to the group for clicks and conversions, you could roll out a new campaign to a sampling of that subset for testing and fine tuning before rolling it out to the entire audience for the campaign.

    Does your company try to identify connector / salesmen in their audience for targeted campaigns (from the book, The Tipping Point)? If you could determine who was forwarding campaign emails with the highest click and conversion from their forwards you could tailor campaigns for the connector / salesmen group for higher pin action (knock down one connector pin and he knocks down three more within his sphere of influence).

  2. Posted March 5, 2009 at 8:15 pm | Permalink

    Jeffry…interesting idea, any thoughts on how this could translate to practical campaign execution?


%d bloggers like this: