A/B Testing Is Alive and Well

Share with:

FacebookTwitterGoogleLinkedInRedditPinterest


A/B testing is the bedrock of a good PPC campaign. It’s so important that I’ve written about it on this blog 46 times. Just last week, I wrote a review of AdAlysis, an A/B testing and multivariate testing tool. And just 2 years ago, I asked, can too many ads ruin PPC ad copy testing?

Spoiler: The answer is yes. Testing too many ads at once creates a myriad of issues, including taking forever to reach statistical significance in all but the highest-volume PPC accounts.

And yet, in their infinite wisdom, Google is now recommending that advertisers forgo A/B testing, and instead run at least 3 ads per ad group. In fact, Google representative Matt Lawson, in an article for Search Engine Land, this week went so far as to claim that using more than 2 ads per ad group is a “foolproof step to excellent Adwords ads.” In the article, he says, “I think the A/B approach to message testing is becoming outdated.”

Wow.

I think what he means is: “At Google, we’d really rather decide what ads are performing best for you. We want you to use the ‘optimize’ ad rotation settings and let us choose which ad to serve.”

That’s right. Google is telling us to forget ad copy testing and just let Google pick the winners.

To a novice PPC advertiser, I’m sure this is music to the ears. Small business owners and in-house marketers who are dipping their toes into Adwords management are probably thrilled to hear that they don’t have to worry about A/B testing ad copy. They can just throw a few random ads into their account, and let Google pick the winner.

Really?

How many successful business owners do you know who let their vendors tell them what products to stock in their stores? When pharmaceutical companies started paying big bucks to get doctors to prescribe their medication over others, the public lost its collective mind. “How dare they buy off the doctors?” If you walked into a clothing retailer who claims to carry multiple brands, and only found Calvin Klein, wouldn’t you wonder about the store owner’s sanity?

It’s called putting all your eggs in one basket. It’s not smart business. And it’s not smart advertising.

I get it. PPC is complicated. And hiring a professional PPC manager is expensive. That’s why many novice business owners and in-house marketers try to tackle PPC on their own. But it’s too complex. PPC is not something you can DIY. You wouldn’t try to fill a cavity yourself. Or replace the roof on your house. Or do your own business taxes. Or elect a president who stands to personally line his pockets using the office. (Wait, did I say that out loud?). The point is, you shouldn’t try to do PPC alone. Years ago, you could. Today, it’s just too complicated. And Google is out there trying to get you to turn the whole thing over to them.

Don’t fall for it. Hire a professional. Use A/B testing liberally. Make your own business decisions.

Julie Friedman Bacchini wrote a great post breaking down the fallacy of the Google article on SEL – go give it a read. And let me know what you think about the Google article. Do you see any merit in it? Are we really heading toward a world where we just let Google make all the decisions? Share in the comments!

Related Posts:

Comments

  1. Google charges money only when someone clicks the ad. So the more accurate retail industry analogy would be where vendors tell stores which products to stock against the commitment that they will take money only when the product leaves the shelf i.e. it’s sold by the store. It so happens that this is a fairly common practice in the retail industry. It’s called “consignment basis”. Many retailers are used to this practice and will find merit in Google’s current offer. TBH, it’s too late to worry about putting all eggs in one Google’s basket. That ship sailed long ago. With or without this offer, what great choice did I have other than Google anyway for digital advertising in general and PPC in particular?

    • Melissa Mackey says:

      But Google wants us to bet on them making the right decision, rather than let us make our own decision. So this would be like clothing manufacturers forcing stores to take clothing that may or may not sell, on consignment.

      I wholeheartedly disagree that Google is the only choice, at least in the US. We have clients for whom we don’t use Google at all, because it’s cost-prohibitive or doesn’t meet the client’s objective. Assuming that there are no other choices out there is common, but untrue. Bing, Facebook, and other social engines work great, as do several programmatic engines. No one should feel that Google is their only choice.

  2. Great article.
    I agree that A/B/C+ testing can slow down our ability to learn from ad performance. This brings up a question for me. To my knowledge AdWords is not a true testing platform: the setting “rotate evenly” does not imply “rotate randomly,” and AdWords retains the right to choose which ad it serves based on perceived profitability based on early data and other factors. Is a winning variation “statistically significant” if the segmentation / grouping was based on choice rather than random chance?

  3. There is an issue with data quantity too. High volume ads generate data quickly, obviously, so A/B/C might work for them (for the record I think it is a redundant idea) But most ads in most campaigns for most advertisers do not generate sufficient traffic to get any usable data in any practical timeframe. So A/B/C dilutes that data even more, stretching the usability unrealistically.

Trackbacks

Speak Your Mind

*