Avoid having a horse in the race: now that doesn’t mean you can’t gamify it with your team and bet on the treatment you think will win. Rather it means step back and see the proposed treatments and elements objectively, letting data and logic dictate which ones make the cut. This should result in a more balanced and thorough array of candidates, which means higher likelihood of statistically reliable results. Bottom line: check your emotions at the door.

Pre-screen your final variations: (in the absence of rigorous qualitative data to guide your decision) at the very least find people within or outside your org who are far-enough away from the idea origination process to have a fresh opinion. Then ask them to give you honest (and preferably anonymous) feedback on #1 what recipe(s) they like and #2 and very importantly why?. Just like sending a manuscript to an editor before publishing, pre-screening could unearth subtle mistakes, missing content and/or possibly give you new ideas to sneak in before launch.

Test A/A/B….: Especially when your test is simply one new treatment against the control, split the control traffic into two audiences which both receive the control recipe. After enough time when A and A have merged in performance, then you can compare AA combined against B. If A and A do not merge, it may indicate sample bias or some other bias is afoot.

Specifics around such a test would vary depending on the details of the marketing campaign. In the attached images (Link 1, Link 2), I have a good example to walk through.

Take a look at the paid search ad (Link 1) - there are some key phrases in place. "Easily Target, Test & Automate" and "Rapidly test and experiment to create .....". Now take a look at the landing page image (Link 2). There is a big disconnect. I am presented with a big overlay but no messaging similar to what I seen in the SEM ad. We could target a Test just to this traffic and easily see if reinforcing this content leads to increases in landing page KPIs.

RELATED: Link 1 · Link 2