A/B Testing and Usability

Casey BartoDespite all the advances in marketing technology, the old standbys of A/B tests and usability testing remain very valuable tools for digital marketers. Here are some tips for each:
A/B Testing vs. Usability Testing: A/B testing is great for conducting marketing experiments. For example, it can help you decide between two subject lines or two calls-to-action. While A/B testing can help answer some of your marketing questions, usability testing can help you ask better questions. If you want to test the functionality of a site or page, usability testing is the way to go.

Or, put simply, A/B testing allows you to conduct randomized experiments between 2 pieces of marketing content, while usability testing lets you evaluate your web pages, etc., in the form of small focus groups. You can listen to and observe users as they’re experiencing your content, design, etc.

Asking Better Questions with Usability Testing: Rebecca Strally at MECLABS has some excellent tips for formulating better questions. These include making sure questions are formulated to help you learn, paying attention to demographics, and never making a business decision based on usability testing alone.

When testing a page, Rebecca recommends avoiding the question “Do you like this page?” There’s really nothing that can be gleaned from the answer. Instead, she says to try asking “Is there anything about this page that confuses you?”

  • If your online usability testing is a paid service, be sure to indicate gender, nationality, etc if you’d like to have that information for your tests.
  • Because usability testing is usually done in several small groups, the groups lack the ability to provide any statistical significance that you could use to make business decisions. Instead, use these results to help you formulate more scientific questions that will make good tests.
  • Consider using an exit survey to get deeper insights into those customers who completed the test and those who did not.

Tips for A/B Testing: If you’d rather focus on A/B testing, here are a few pointers for boosting your tests.

  • Be sure to keep checking your data throughout your test. Codes can get changed and data can become inconsistent during testing.
  • Try not to introduce too many variables at once so not to cloud the results.
  • Understand that there are no failures – only lessons. Use what you learn from your testing, document it and allow it to positively influence future outreach.
%d bloggers like this: