Tips for Successful A/B Testing

Emily HaaseA/B testing is an excellent way to gain insight into customer behavior and increase conversions. With the right tools, you can create and launch an A/B test in a matter of minutes, and easily have the winning piece of content in market. It’s also possible to do all of this without bothering your IT team.
Concentri’s A/B testing tools are extremely easy to use. Simply select the “A/B test” tab when creating new content and enter your test content. Once you’ve entered your test content, just click the “start test now” button and you’re off and running. What’s really cool is once the statistical significance has been reached on your test, it’s easy to select the winning piece of content and send it to the rest of your list. (Note: After you test and select the content you prefer, be sure you remember to set up and schedule the remainder of your send.)

If you’re a Concentri user, you’ve got everything you need at your fingertips to make the most of your A/B testing and direct digital marketing initiatives. If you’re not a Concentri user, learn more here. In addition to having great tools, you also need a sound strategy.

Here are some tips for making the most of your A/B tests every time:

Allow Enough Time: While it might be tempting to quickly run an A/B test in order to have your content in market faster, resist. Always allow a minimum of 24 hours to test your content. If you only allow your test to run for a couple of hours, you’ll likely miss out on valuable consumer information. For instance, a consumer might not check their personal email during the workday, and will miss out on participating in the test when they check their email during the evening. Or a customer’s email client might hold your email, delivering it after the test has been completed. Don’t rush.

Don’t Test More than 4 or 5 Pieces of Content at a Time: Most everyone learned in high school to only test one hypothesis at a time. The same concept applies here. If you test large amounts of content, it will be difficult to keep track of cause and effect relationships as well as what you are learning.

Make Sure You Have a Statistically Significant Sample Size: Don’t just split your list in half and run a test on each half. Choose a statistically significant sample size for the size of your list (remember, Concentri does the math for you and automatically detects the winner). If, for example, you have a list size of 1 million, you might send a test to 10 percent of the list. If your list size is 100, however, sending to 10 percent of that list won’t net you the results you need to have a successful test.

If you have more A/B testing questions, drop us a line. We’re always here to help.

Thanks to Joe McGarvey, Onboarding Specialist here at Knotice for contributing to this post. Stay tuned for more from Joe!

4 Trackbacks/Pingbacks

  1. […] Email: 4 Quick Tips Before You Send Someone once explained that personal email is like a handgun, and the send button is the trigger. You have to be sure your message is clear and correct because once you’ve “fired off” that email, as you can’t take it back. And that was just referring to personal emails. For direct digital marketers, the stakes are far higher. When your scheduled send command delivers messages to hundreds of thousands of people at once, it’s nothing to take lightly. To take the gun analogy even further, once email campaigns are “locked and loaded” into your system, that handgun quickly becomes an automatic weapon. This compounds the need for scrutiny when proofing and testing. For details on testing, check out Emily’s post here. […]

  2. […] best way to avoid them is with thorough testing. Check out these posts for more testing tips here, here, and […]

  3. […] When most of us think of making improvements to our campaigns, testing comes to mind. (Check out these A/B testing tips.) You’ve thought about testing copy, images, layout, etc. But have you also considered testing […]

  4. […] some additional A/B testing tips, take a look at these posts from Emily and Dutch. Share this:Like this:LikeBe the first to like this. This entry was written by Joe […]

%d bloggers like this: