Research by Forrester released early in October indicates that display advertising spend will grow 17% annually from 2012 to 2017. As display continues to grow, so does advertisers’ needs to measure the incremental value gained from display efforts across other digital tactics.
ClickZ’s article A/B Testing Done Right…and Wrong outlines several of the most important factors advertisers need to take into account when measuring display lift.
- Volume – Statistically significant sample sizes are massive. For most advertisers it is not feasible to run an A/B test because of the costs and time needed to generate a large enough sample size.
- Sampling Bias – This is nearly impossible to avoid because of the fact that there are some groups of people online that are less likely to be include in the test group than others.
- External Factors – It’s important to remember that all efforts run outside of you’re A/B test could affect your test and skew results.
- Cost v. Benefits – Before you start any A/B test you need to consider the benefit the results will show and the cost of effectively conducting the test.
Check out the full article for more details on each factor. At the end of the day, advertisers and marketers need to set expectations up front regarding test results. If you move forward with an A/B test where you may not be able to garner the volume needed, it is important to know this going into the test and keep it in mind when reviewing test results.
