I get asked frequently if Market2Lead supports A/B testing of email campaigns and the answer is “yes, of course,” but sometimes I wonder if the people have thought through the process.
You can dissect the average email campaign down into 6 broad
areas that can influence the outcome as shown in the graphic above. In my
August 28, 2009 post on effective campaigns I discussed the value of a high
quality list as the primary determinant of the effectiveness of a campaign, so
we won’t repeat that here. Instead I want to discuss how one would best go
about A/B testing campaigns for the 5 remaining items.
Select the A/B Testing Outcome To Test
In B2B with long sales cycles, and multiple touch points, it would be silly to A/B test email campaigns for closed won opportunities, but in B2C this might be the perfect answer. In B2B email campaigns you are probably more likely to constrain yourself to opens, clicks, and offer downloads. In B2C with short sales cycles, you might consider “added to basket”, or purchases as the outcome you are testing.
A/B Testing Requires Random List Segmentation
To be an effective A/B test, the other factors beyond the one you are testing must be held constant or, in the case of segmented lists of people, be made random. So the first question should be, how does one produce random lists:- Allow the marketing automation system to randomly pick people from your segmented list for either the A or B treatment. To the best of my knowledge no marketing automation vendor offers this today.
- Manually produce two lists (A and B) using some random mechanism; you could use the odd numbered contact ID numbers for the A list for instance. Trying to create random lists by selecting letters found in names or emails is not as random as you might think! But if the contact ID numbers are assigned randomly when new leads are created or imported, it is a great way to go.
A/B Testing with One Program and Two Lists?
Some folks believe that it is simpler to use one program with two random lists, and have the functionality built in to apply the appropriate A/B subject line, or A/B email body, or A/B offer, or A/B Landing Page, or A/B Form to the A or B target list. In the reporting, the system would then have to be smart enough to produce two different reports on clicks, opens, downloads etc based on the A/B variable that was chosen. I find this too complicated.
I much prefer to create the program for A, copy it, point it to list B, change my one variable (subject line, or email body, or offer, or landing page, or form) and launch both programs. I can do this in two minutes and they each have their own results, it is easy to interpret, and easy to copy these programs for future tests.
You Don’t Need Dedicated Functionality for A/B Testing
The bottom line is that you can do some very creative A/B testing or multivariate testing as long as you have a means to create random lists. Everything else is easily done by copying the program and changing your one variable, and the reporting will make much more sense to you and everybody else.
So, an exercise for the reader: How would you do A/B/C testing since 3 does not divide evenly into 10, so our trick of using odd versus even contact IDs won't help...
-Kevin

