Ad Testing: Conversions Per Impression

Ad copy testing is one of my favorite aspects of PPC. Where else can you get so much great data about what people respond to, so quickly?

The challenge in ad copy testing isn’t so much writing the ads, it’s deciding how to pick a winner. And there are many different opinions on how ad tests should be decided. If it were up to Google, they’d pick click-through rate as the deciding metric – as evidenced by their default “optimize” option:

ad rotation
(I hope all my readers know that “optimize for clicks” is NOT the ideal setting for most advertisers!)

Others may say total conversions should decide the winner in PPC ad testing; still others may vote for conversion rate or cost per conversion.

Arguments can be made for all of these options. An argument can even be made for CTR, if traffic, and not conversions, is your goal. I’ve optimized accounts using most of these metrics. And I’ve often seen that the ad with the worst CTR has the best conversion rate. Sometimes you actually want to discourage clicks from the wrong users, so a low CTR may be a good thing.

But my favorite way to analyze ad tests is using conversions per impression.

I first learned about conversions per impression from Brad Geddes in a blog post several years ago. It’s a revolutionary concept, and one that works well especially for lead generation. (If you’re doing ecommerce and measuring ROAS, you may want to stick to ROAS as your testing measurement instead of conversions per impression, as Brad outlines in this post).

The conversions-per-impression metric often serves as the tie-breaker when one ad has a higher CTR and the other has a higher conversion rate. Here’s an example:

conversions per impression example a

This image is from AdAlysis, another great tool that Brad developed. AdAlysis highlights winning metrics that meet a certain confidence level. In this test, you can see that one ad is winning for CTR at 99% confidence, but the other metrics are less than 90%.

I added green highlighting for the metric that’s higher in each of the conversion columns. Although these aren’t winners yet, you can see that the first ad has a higher conversion rate. If you were measuring conversion rate alone, you might be tempted to pick the first ad as your winner.

But look at conversions per impression and cost per conversion. Both of these numbers are better for the second ad. If you picked the first ad based on conversion rate alone, you would have possibly picked a loser.

At a minimum, you’ll want to wait until the conversion metrics reach statistical significance, especially since impressions are pretty different between the two ads (the first ad has half the impressions of the second, even though they launched on the same date). But it looks like the second ad is in the lead here.

Here’s another example:

conversions per impression example b

This is an interesting one. These ads have nearly identical impression numbers. The first ad is winning for CTR, but the second is winning for both conversion rate and cost per conversion. You may look at this and say “well, I bet the second ad is doing a better job of weeding out unqualified visitors. I’ll pick that as my winner.”

You’d be wrong. Look at conversions per impression here. The numbers are nearly identical, but the top ad’s number is slightly higher. This ad may end up doing better because of the higher CTR.

Remember, every impression is an opportunity for a conversion, as is every click. Conversions per impression takes both CTR and conversion rate into account, and allows you to maximize conversions per opportunity. That’s why I like to use this metric, especially for lead gen where revenue doesn’t play into the equation.

If you’re not using AdAlysis, you can calculate conversions per impression manually – just divide total conversions by total impressions, and run the numbers through a statistical significance calculator. (And if you’re not using AdAlysis for ad testing, what are you waiting for? It’s inexpensive, and worth every penny.)

Whatever metric you use to determine ad tests, make sure you use best practices for picking the winning ad. Don’t guess. The stakes are too high to risk a wrong guess!

What metric or metrics do you like to use to evaluate PPC tests? Have you tried conversions per impression? Share in the comments!

Related Posts:

How To Pick The Best PPC Ad

In a recent thread on Reddit, a user asked how to pick the best PPC ad from an ad copy test. The user mentioned two ads, one that had 6 conversions and another that had 4. They asked which ad they should roll out with.

If you’ve read my blog for any length of time, I hope your answer would be “none of them.” Ads with single-digit conversions usually do not have enough data to determine a true winner.

Most of the commenters pointed this out, but there were enough responses suggesting that the user guess at a winner that I decided I need to write yet another post on ad copy testing. Here’s a guide for how to pick the best PPC ad.

Ad Testing Overview

About a year ago, I wrote a comprehensive post on ad copy testing. This should be your starting point for setting up and evaluating PPC ad copy tests.

Give It Enough Time

Tempting as it may be to decide on a winning ad early on, it’s a mistake. Even very high volume accounts should accumulate at least a week’s worth of data before deciding on a winner. And definitely don’t declare a winner because one ad had 6 conversions and the other had 4. While PPC provides a lot of immediate data, ad testing is too important to let a few days determine your results.

Test Smart

There are so many things you can test in PPC: titles, descriptions, calls to action, display URLs, landing pages….. it’s tempting to test a whole bunch of them at once. Resist the temptation. Just because you can test something doesn’t mean you should.

Don’t Run Too Many Ads at Once

I see this with nearly every account we inherit: ad groups with as many as 10-12 ad variations running at the same time. Rarely does that make sense, even in large, high volume accounts. It just takes too long to determine a winner – and meanwhile, you’re spending money on losing ads. Too many ads can definitely ruin ad copy testing.

Use My Ultimate Cheat Sheet

My Ultimate Cheat Sheet on PPC Ad Copy is a handy reference when setting up PPC ad copy tests. Use it to make sure you’re setting up quality tests.

Read The AdAlysis Blog

I’ve made no secret of my love for the AdAlysis tool. It’s saved me countless hours in evaluating ad copy tests. The AdAlysis blog is a treasure trove of articles on smart ad copy testing. Give it a read and subscribe today.

What are your favorite tips for choosing winning ads? Share in the comments!

Related Posts:

The Ultimate Cheat Sheet on PPC Ad Copy

When people think about PPC, they often think about keywords first. But good ad copy is just as important to a successful PPC campaign. With only 90 characters to work with, writing good PPC ad copy is harder than you think.

What is the goal of each ad group?

Think about the goal of each ad group. Is it to sell products? Generate leads? What specifically are you offering and what do you want people to do when they get to the landing page? Your goal will help you craft ad copy that encourages users to do what you want them to do.
Read more: Ideas Are Not Strategy

Write the call to action first.

Few PPC pros write ads this way, but they should. Writing the call to action first forces you to craft ad copy around it. It also ensures enough space to say what you need to say! Some calls to action are long -“Download The White Paper Now,” for example – so you need to make sure you have enough space, or at least know you need to write a shorter call to action.
Read more: PPC Ad Copy Creation: Where To Start?

Research the competition.

You can use a tool like SEMrush or AdGooRoo, or you can just perform a few ad hoc searches on your keywords. Find out what your competitors are talking about and what they’re offering. If they all offer free shipping, you’ll need to think seriously about doing the same thing. Sometimes, researching the competition yields ideas on how to differentiate your business from all the others in the space. Are you the only company that allows purchases without a credit card? Say so! Do you ship faster than others? Put that in your ad copy.
Read more: 3 Sneaky Ways To Bid On Competitor Keywords

Determine your unique selling proposition (USP).

What’s unique about your company? Why should people buy from you? This is your unique selling proposition, the differentiator for your business. By the way, slogans such as “Just Do It” aren’t USPs. There’s a place for slogans, but not in ad copy. Save them for callout extensions and put a true USP in your ad copy.
Read more: Why PPC and SEO Engagements Fail

Include elements to encourage conversions.

In PPC, ad copy must do two things: stand out to encourage clicks, and make the offer clear to encourage conversions. Include elements such as numbers, keywords, and urgency statements (“Limited time offer!”) to encourage users not only to click, but to convert.
Read more: Pay-per-click: 5 Tips for Successful Ad Copy

Test, test, test.

Your job doesn’t end once you’ve written one ad. It’s just started. Write at least 3-4 ads for each ad group. You’re not going to put all 4 ads into market now, but you’ll need them for testing. Once one ad wins, pause the loser and rotate in another ad you’ve already written. One of the great things about PPC is the ability to test ad copy. Use it! Learn from it!
Read more: PPC Ad Copy Testing: 2015 Edition

The Ultimate PPC Ad Copy Cheat Sheet

PPC Ad Copy cheat sheet

Download the cheat sheet in Excel here: The Ultimate Cheat Sheet For PPC Ad Copy

What are your ultimate PPC ad copy writing tips? Share in the comments!

Related Posts:

PPC Ad Copy Creation: Where To Start?

Last week’s post on ad copy testing prompted some thought-provoking discussion in the comments. My friend Jerry Nordstrom asked:

When you are creating ad variations what is your primary driver of each iteration?
Keyword intent? Keyword volume? Overall ad intent? Dynamic or Mobile only ads?

This was such a good question that I thought it deserved its own post. Here’s how I approach PPC ad copy creation.

Make sure your ad groups are tightly themed.

This may sound like it has nothing to do with ad copy, but it actually has everything to do with it. If your ad groups contain large groups of diverse keywords, writing relevant ad copy is going to be darn near impossible. With small, tightly themed ad groups, the ad copy nearly writes itself.

Let the overall intent drive your call to action.

Even now in 2015, I see a lot of ads that lack a strong call to action. They’re either full of semi-boring facts and features, or they’re trying to be funny and creative – yet they forgot the most important part, which is to tell the searcher what you want them to do! Go back to your campaign goals, review your landing page, and then tell people what you want them to do in the ad copy.

Incorporate the keyword into the copy whenever possible.

I’ve seen this done well, and done poorly. Keywords for the sake of keywords, or excessive use of dynamic keyword insertion, puts ad copy into eBay territory. But I do recommend getting the keyword into the ad if you can. It may sound old-school, but getting that bolded copy you get when the keyword shows up in the ad is still valuable.

Take keyword volume into account.

This isn’t a primary driver for me, but I do usually focus on the highest-volume keyword in the ad group when writing PPC ad copy. It’s efficient, for one thing; and it tends to perform best.

Take device and user actions on said device into account.

You probably know by now that the clients I work on are primarily B2B lead generation advertisers. Many of our clients get most of their leads over the phone. So when I’m writing mobile ad copy, I make sure to use something like “Call Us!” as a call to action. For desktop ads, I’ll say “Learn More” or “Get The Free Report” or something along those lines.

Some advertisers will find that mobile users are looking for something totally different from desktop users. Take “dominos pizza,” for example. Desktop searchers might be looking for a menu, a location nearby, or even information on the company headquarters. But mobile searchers probably are starving and want to order a pizza right now. Craft your device-specific PPC ad copy carefully.

All of the above are just starting points, mind you. Always test and refine – what works for me, or even one campaign in an account, may not work in another. Test, test, test!

How do you craft PPC ad copy? What’s your #1 go-to tip? Share in the comments!

Related Posts:

PPC Ad Copy Testing: 2015 Edition

I love PPC ad copy testing. It’s one of my favorite aspects of PPC. Where else can you systematically test different elements of ad text and learn what works the best – all in a relatively short period of time?

Like with many things in PPC, ad copy testing has evolved over the years. I’ve written about it a few times, and every time I look back at those articles, I realize how much has changed – and how much I’ve learned since then.

Here’s my 2015 approach to PPC ad copy testing.

Mind your campaign settings.

Google continues to insist that “optimize for clicks” and “optimize for conversions” are the “ideal settings” for ad rotation:

ad rotation

They’re still wrong. I always recommend “rotate indefinitely” unless your goal is site traffic (which it shouldn’t be).

Create a bunch of ads for each ad group.

I used to create 2 ads, run with those, and then scramble to think of new copy to test. I started my PPC career in an in-house setting, so winging it with ad testing is usually fine. When I started working in an agency, with enterprise-level clients, I realized that winging it wouldn’t cut it. Clients need to review and approve ad copy before it runs.

Now, I create as many variations as I can think of: at least 5 per ad group and 10 or more if I can. Think of everything you might want to test. Here are some ideas:

Testing Matrix
Send the whole shebang to your client or boss for approval. It’s much more efficient this way.

Test only 2 ads at once.

At this point, you may be thinking “wait a minute – why did I create 10 ads if I’m only going to run 2?” Simple: You now have new ads at the ready to swap in when a winning ad emerges.

By testing only 2 ads at once, you’ll reach statistical significance much sooner than you would if you test 5-10 ads all at the same time. You’ll also learn what’s working: did DKI perform better? Did one call to action work better than another? Then you can use this knowledge in your upcoming tests. The result? Higher conversion rates, faster.

Create an ad testing matrix.

I illustrated how to do this in an article for Search Engine Watch, so I won’t repeat it here. Suffice it to say that systematic, rather than random, ad testing is best.

Review results regularly.

How regularly depends on how much traffic you get. For high-volume ad groups, you might want to review ad test data weekly or even daily (although weekly is usually best to get a full view of performance over days of the week, especially weekends). For lower-volume ad groups, monthly may be too frequent. Find the cadence that works for you.

As for how to review results, my process has evolved dramatically over the 13 years I’ve been doing PPC. I used to use spreadsheets and manual summing, and then I’d pick the ad with the highest conversion rate. That’s problematic for a number of reasons, the biggest being the lack of statistical significance.

Then I started using statistical significance calculators, some of which have come and gone (anyone besides me still lamenting the demise of SuperSplitTester?). My current favorite manual calculator is the one from Visual Website Optimizer. Download it and use it!

My very favorite ad testing tool, though, has to be AdAlysis. It’s a paid tool, but you’ll recoup its modest cost many times over in the hours you’ll save in calculating significance. Check out this geeky goodness:

(click to see the whole thing)

At a glance, you can see which of your ads has reached significance on several metrics: CTR, conversion rate, conversions per impression, cost per converted click, conversion value/cost, and conversion value/impression. Whew! It also shows the impact on campaign performance if you pause the losing ad. Projections: done.

You can also do multi-ad group testing with AdAlysis. So if you have multiple ads with the same headline or call to action across ad groups, you can lump them together to gauge the overall impact of each element. I’ve done account-wide call-to-action testing with AdAlysis, for instance.

Its only downfall is that it only works for Adwords at the moment. So you’ll still need to analyze Bing and social ads manually.

(I’m not paid for this endorsement, by the way – I just love the product!)

Rinse and repeat.

This is my final tip in all the PPC ad copy testing articles I’ve written, and even in 2015 it needs to be said. So many ad tests are set up and never analyzed. Don’t let that happen to you! You’re better off not testing – at least you won’t see how much money you wasted on an underperforming ad.

Always review your test results and, at a minimum, pause the losers.

What are your favorite PPC ad copy testing techniques? Share in the comments!

Related Posts:

Why Sitelinks Are A PPC Worst Practice

A few months ago, one of my coworkers asked me a thought-provoking question that I’ve been ruminating on ever since. She asked, “Does Google’s increased push on the use of sitelinks contradict their best practice to make ad groups as specific as possible and to drive users to the most relevant page? Say I’m advertising blue widgets. Long-standing best practices would be to have a very specific ad group pertaining only blue widgets and using my blue widgets page as the destination URL. Now instead of just being able to send them to my blue widgets page, I’m being pushed to include less relevant pages to keep my ad at the top of the page – Widgets, Widget History, Widget FAQs, etc. If your campaigns and ad groups are properly organized, sitelinks are only useful in limited circumstances.”

I thought this was an interesting perspective – one that I agree with. With our B2B clients, I usually don’t use sitelinks, for this very reason. The client has specific goals for each product or service, and we structure our campaigns and ad groups accordingly. The client doesn’t want us sending traffic to other pages within their website – these pages may not be optimized for conversion, or they distract the visitor from taking the action that the client really wants them to take.

Also, there are times when 6-10 or even 1-2 relevant links besides the landing page just don’t exist. Again, the client has a specific product or service they want us to promote. Maybe they even have a budget dedicated to that product or service. They not only don’t have other pages for us to send traffic to, but they don’t want us using their budget for that traffic!

This problem is more common for B2B advertisers, to be sure. I discussed it with Jeremy Brown in a post back in 2012.

This isn’t the first time I’ve covered the pitfalls of sitelinks. Back in 2011, I wrote a post for Search Engine Watch about the not-so-great aspects of sitelinks. While 3 years is an eternity in search, and Google has fixed most of the issues mentioned in that post, there are still shortfalls. Conversion tracking is still a challenge.

And Google doesn’t make it easy to see how individual sitelinks are performing. Take a look at this example:

sitelink data

At first glance, it looks as though the Contact Us sitelink has driven 8 conversions. Not so fast:

this vs other

In reality, no one is clicking on “Contact Us” – they’re all clicking on the ad itself.

This isn’t unusual, but to new PPC manager, or to clients looking at their own data, it’s misleading and confusing to say the least.

But I digress. I’m not the only one who thinks sitelinks might just be a worst practice. Andrew Goodman, in his famous rant “Why I Hate Sitelinks,” lists 11 reasons why he believes sitelinks are problematic. #1 on the list really resonates with me: “Where is the testing? Where are the key performance indicators (KPIs)? It’s impractical and/or irrelevant to test them; you can’t get actionable feedback.” Indeed.

I’m not totally anti-sitelinks. Sitelinks, and ad extensions in general, are a great way to take up more screen real estate. For advertisers with a robust catalog of related products and pages, sitelinks make a lot of sense. But they’re not for everyone, especially those with tightly-themed ad groups or those with only 1-2 relevant landing pages.

What do you think? Are sitelinks the love of your PPC life, or are they a worst practice? Share in the comments!

Related Posts:

The Most Important Element of PPC Ad Copy

One of my favorite tasks is writing PPC ad copy. I love the challenge of squeezing as much goodness as I can into 90 characters. I guess it’s in my blood after all these years of writing short, concise ad copy.

Recently, I was asked, “What’s the most important element of PPC ad copy? Is it the headline, the first description line, or what?” Of course, my kneejerk response was “It depends,” which is really the answer to just about any question related to PPC.

Still, the question got me thinking. Is there one element of PPC ad copy that’s more important than others in most cases?

If I had to give only one answer, I’d say the headline. After all, it’s the only part of a PPC ad that’s underlined, and if you include keywords in your headline, they’ll be bolded:

headline

But as I look at the ads in that screen shot, I start to think that other elements might be equally important.

Sitelinks

Surprisingly, the first ad on the page above isn’t using sitelinks, but the other two are. The Amazon sitelinks are great – they’re all relevant to the category (Kitchen & Dining, Small Appliances, Kitchen Tools & Gadgets) and season (Off to College). Target falls down a bit by focusing entirely on college back to school. The Magic Bullet is a product I’d be interested in, but I’m long past the college dorm move-in days.

Still, the use of sitelinks takes up more screen real estate, so that’s an important element.

Reviews

See how the review extension makes the Amazon ad stand out? Those nice red 4.5 stars are visually appealing to say the least. If you qualify for review extensions, they’re definitely important.

Call Extensions

2 of the 3 ads above include phone numbers. One of them is relatively local – although Jackson, MI is at least 30 minutes away from me, and there’s a Target within 10 miles. Still, the phone number can be another important element of a PPC ad, in addition to a great way to boost conversions.

Location Extensions

That Target ad also includes a location extension. While these make ads stand out on the page, I hesitate to say they’re the most important element because most advertisers cannot track in-store sales back to a PPC ad. And in the case of this ad, even if I decided to buy at Target, I wouldn’t drive half an hour to that Target – I’d go to the Target closer to me. Either way, location extensions do help ads to stand out.

All of these elements are great, but you have to be in a premium position to take advantage of most of them. What about when your ad ends up in the right rail, like some of the ads below?

right rail

When ads appear in the right rail, you have fewer options. And of course you don’t have control over whether your ad shows up on the top or the right. So you need to make your ad copy work no matter where on the page it happens to appear.

This is where basic ad copywriting comes into play. A good headline, ideally with the keyword included, is going to be critical. Including the keyword in the body of the ad is also important. But are other elements just as important?

Call to Action

A lot of advertisers forget to include a strong call to action at the end of the ad copy. That’s usually a mistake. It’s a good idea to tell searchers what you want them to do when they get to the landing page – and ideally it’ll be to convert: buy something, download a report, etc. So I’d have to say that the call to action is nearly as important, or maybe even more important, than the headline.

Copywriting Tips and Tricks

There are a few tricks that I’ve learned that work no matter where in the ad they appear. One is the word “free.” “Free” is the magic word when it comes to PPC. People absolutely love it, and ads with the word “free” are almost always going to get better results than ads without.

Another element is just a single character: the exclamation point. You’ll notice that ads across verticals, even in B2B verticals like the above example that tend to avoid marketing hype, use exclamations. There’s a good reason for that – it works.

I’ve run tests where the ads were identical except that one had an exclamation and one didn’t. The ad with an exclamation way outperformed the ad without.

So What’s The Answer?

You know what I’m going to say, don’t you? The most important element of PPC ad copy is the element that drives the best results, of course!

What’s your take? Is the headline the end-all? The call to action? Something I didn’t list? Share in the comments!

Related Posts:

Don’t Do What I Tell You To Do

As a mom, it pains me to say that. I’m constantly harping on my teenage twins to do what I tell them to do – and half the time, they do the opposite, thus frustrating the heck out of my husband and me.

So how can I bring myself to tell you, PPC friends, not to do what I tell you?

This week was a busy one in the SEM world with Pubcon Vegas, which wrapped up yesterday. Twitter was alive with Pubcon tweets – so much so that it was hard to keep up.

But there was one Twitter gem from Pubcon that stood out in my mind: “When a speaker says ‘do this’ you should hear ‘test this’.” Tweeted by Ryan Jones and attributed to my friend Brad Geddes, the wisdom stuck with me.

PPC is a smorgasbord of testing goodness – a veritable paradise for a numbers geek like me. Attendees at search conferences will come back with pages of notes full of new ideas to implement in their PPC campaigns. PPC newbies, especially, will be tempted to take the word of the “experts” as the law of the land. Heck, I’m one of the experts that will be speaking at SES Chicago next week, imparting wisdom upon PPC rookies.

Admittedly, there are some basic rules every PPC manager needs to follow. But in general, don’t blindly do what we tell you to do! What works great for one campaign may be a huge flop in another. Nearly everything that’s worth doing in PPC is worth testing first. That’s what I love about PPC – the ability to test just about anything and learn from it.

So next week at SES, when a speaker says “do this,” make sure you hear “test this.”

(Hey, maybe that’s what my kids are doing – testing it for themselves….)

Related Posts:

  • No Related Posts

PPC Ad Testing: “Can” vs. “Should”

I love PPC for a number of reasons. It’s fast and effective, it doesn’t require a ton of money to use, and there are few other marketing channels that make ad testing so easy and effective.

But as much as I love PPC ad testing, I feel the need for caution. Like a kid in a candy store, PPC managers often have trouble choosing what to test. Just because you CAN test 20 things at once doesn’t mean you SHOULD.

One of the best things about PPC is the fact that you can run an almost unlimited number of tests. You can test 10 different ad variations if you want to. If you’re using display, you can even test text ads vs. image ads. Within the image ads bucket, you can test rich media, animation, several different sizes….. You get the picture.

I’ve found that new PPC advertisers (and clients) see this buffet of choices and try to pile one of each on their small appetizer plate. They set up a multitude of ad tests right away, before they’ve launched a single campaign or gathered one iota of data. The thinking is, “we’re not sure what will work, so let’s try them all!”

To an extent, they’re right. With a brand new advertiser and a brand new campaign, you really don’t know what will work. Because PPC generates a lot of data so quickly, in some ways it’s logical to test every option so you can get that data as fast as you can.

But let’s think about this for a second. Way back in business school, we learned about a little thing called opportunity cost. Opportunity cost takes into account a few key factors.

Time.

The mere task of setting up 10 or more ad tests can be daunting, even to an experienced PPC manager. What copy will we use in each variation? What exactly are we testing? Even with a tool like Adwords Editor, it takes time to set up each variation – and it’s compounded by the number of ad groups you’re working with.

There’s also the issue of the amount of time it takes to get results. For simplicity’s sake, let’s say you need 100 clicks on each ad for statistical significance, and you estimate it’ll take a week to get that many clicks in each cell. If you’re running 2 ads, that’s 200 clicks – but if you’re running 10 ads, that’s 1,000 clicks. You’ve just turned a 1-week test into a 5-week test. And in the meantime, you may be running an ad that’s losing money – which you won’t find out for 5 weeks. Yikes.

Margin of error.

The more information you’re working with, whether it be text or data, the higher the chance of making a mistake. How many of us have copied and pasted the wrong ad copy into an ad group? How many of us have made a crucial typo in ad copy? (Side story: Years ago when I worked in newspaper classifieds, we ran a real estate display ad for an open house at a $300,000 home, complete with a lovely photo of the sprawling manse – and put $30,000 in the ad copy. Needless to say, the open house was mobbed with unqualified buyers. We got a pleasant call from the REALTOR the next morning that I can’t repeat here. And no, this wasn’t my error. But it sure was memorable.)

The margin for error is even greater when it comes to analyzing test data. Running statistical significance on all the permutations in a huge multivariate test is not a ton of fun – and a math mistake can cost thousands of dollars.

Money.

By now it’s pretty clear how a complicated test can cost you actual dollars and cents. If a test takes too long, you might be paying for losing ads for weeks or even months. And if you make a mistake in ad copy or in analysis, that’ll cost you, too.

The moral of the story is, just because you CAN test all kinds of fancy things in PPC doesn’t mean you SHOULD test them. Using a systematic approach is much better (and easier) in the long run.

Related Posts:

  • No Related Posts

The Dos and Don’ts of PPC Testing, Or, “Days Are Not Data”

Last week, I wrote about the new Adwords option to optimize for conversions, and I talked a bit about testing principles. In my experience, PPC testing is one of the most misunderstood aspects of PPC. I’m always surprised by how many advertisers don’t test at all – which is, to be blunt, a huge missed opportunity. Yet many advertisers just put one ad variation into each ad group and let it go, never knowing whether it’s really the best variation or not. Wouldn’t you rather know what ad message generates the most traffic & conversions for your money?

I’ve also frequently seen inexperienced advertisers overreact to normal daily variations in performance. Even the best-managed campaign will have ups and downs on any given day; traffic, conversions, CTR, and any other metric can vary, sometimes wildly, on a day to day basis.

At a recent SES, my good friend Andrew Goodman made what was probably an offhand comment in one of his presentations, yet it stuck with me because it was so profound and true: “Days are not data.”

I love it – it’s brilliant. And true. You don’t need to be a statistician to realize that day-to-day fluctuations do not represent statistical significance in any way. Yet I regularly hear from clients, and even PPC managers-in-training, when results go up or down in a day’s time. These fluctuations, especially at the keyword level, should almost never be cause for alarm.

So how do you know when you have meaningful data? Here are some rules of thumb, based on best practices and years of experience.

Look at a large enough set of data.

If an ad group or keyword got 2 clicks yesterday and 10 clicks today, I can tell you right now that you don’t need to worry about it. Not only are the total numbers too small to be significant, at a minimum you should be looking at week over week data. I’ve written about dayparting recently, and the whole premise behind dayparting is that performance varies from day to day. So don’t make any judgments until you have at least a full week of data, if not more.

Another good rule of thumb is to make sure your data set has at least 100 clicks. You may need more than a week to amass that much info. Be patient – it’s worth the wait to know you’re looking at significant data.

Don’t guess – use statistical tools.

At SMX Advanced last year, I spoke about evaluating PPC tests using SuperSplitTester, which is my favorite easy-to-use statistical tool. But you don’t have to use that one. Just use any tool – but make sure to run the numbers and don’t guess. I’ve used SuperSplitTester enough to guess the winning ad correctly a lot of the time, but not all the time. Don’t guess – your clients and/or employer will thank you.

Evaluate test data systematically.

Yeah, that sounds like a stats prof talking, but what I really mean is set a schedule to review test data, and stick with it. We’ve found that a monthly review is enough for most advertisers when you’re talking about ad tests – even our high-volume clients often don’t reach statistical significance before a full month has passed. Having a set schedule to follow not only ensures the work will get done, but also ensures that the test periods are relatively similar from month to month.

And if you’re really freaked out, only change 1-2 things at once.

One of our clients recently shifted their business goals and strategy, which required a pretty big shift in their PPC campaigns, as well. We launched 3 new campaigns all at once. (I don’t always advise this, but in this case it made the most sense.) When I checked it the day after launch, spend had gone through the roof. Like a good PPC tester, I didn’t panic – but I did lower the campaign daily budgets a bit, just to improve my comfort level. What I didn’t do is go in and start pausing keywords and ad variations, and making a bunch of bid changes – it’s too early for that. The point is, if you’re freaked out, do make a couple changes, but then give it at least a couple days to gauge the effect.

Using systematic, smart testing processes will really pay off in PPC campaign success, I promise!

Related Posts: