Last week I had an opportunity to attend a presentation of Professor Steve Tadelis at Melbourne Business School. It was part of ‘Talking Data’ series and Prof Tadelis shared his experience of a large scale study he and his colleagues conducted for eBay few years ago. The study is called “Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment”.

It was indeed a large scale experiment where researches investigated how effective is paid search – ads that appears if you type a search term in search engine (nowadays in Australia it is mostly Google Adwords). You can find reference to the study paper at the end of this post, but here I’ll present a short, simplified version of what the researches did and what they found.

ROI of PPC advertising with your own brand or company name as a keyword

eBay decided to find out, does it make sense to pay search engines (namely Google and MSN/Bing and Yahoo!) for company name as a keyword. What happens if eBay stops doing it?

So they initially halted the advertising on brand term (eBay) on MSN and Yahoo! for a while. What happened? There was no drop in total traffic from searches, visitors just started to arrive through unpaid, organic traffic. Then the researches continued the experiment with Google, moreover they stopped brand keyword ads and then started them again. Similar with MSN/Yahoo! total traffic virtually for the period of no ads not suffered, organic clicks compensated the lost of paid one. The chart below illustrates what happened.

Paid Search advertising on brand keywords - eBay experiment

A critical difference is that eBay didn’t pay for organic traffic, while paid search advertisement costed the company millions of dollars. So for eBay paid search advertisement produced negative ROI!

It pretty much corresponds with common sense: if your company is already number one in organic search results why spend money on paid search ads here?

Non-brand keywords

The researched went further and investigated ROI of paid search advertising on non-brand keywords like product or category names. A simple, naive way to calculate ROI on paid search ad (net profit on from sale on click cost) has a serious flaw – it doesn’t take into account the fact that some customer would make a purchase without clicking on paid search ad, arriving to brand website via alternative avenues such as organic search result or making extra efforts to type address in the browser. So eBay conducted controlled experiment where they disabled paid search ads on some product keywords in certain geographical location for certain time. It was done in pretty sophisticated way, so if you are really interested in the mechanic – have a look into the paper listed in the references.

The results were shocking, for eBay paid search ads produced in general negative ROI!

If we go even deeper, paid search ads made positive ROI, but for certain customer segments:

  • New customers, who haven’t used eBay before
  • Irregular customers, who have used eBay a while ago

For more or less regular customers, the ROI was negative.

Important note

If after reading this you decide to stop advertising at all, slow down. What works for eBay may not work for you, eBay is a huge company with well-known brand, loyal customer base and good organic ranking. If you are a smaller business, have less brand awareness and poor organic SEO, you may be in a different situation.

What is important however is not blindly buy in what advertising sales and account managers tell you and try to experiment and learn from that experiments, there is a high chance that you burn less money and get a better ROI.

What’s going on in Australia

I’ve conducted a research of website performance of top 100 Australian online retailers for my eCommerce consulting business and in parallel decided to check, do they bid on their brand name. What I have found pretty much shocked me – 75% of market leaders are still purchasing brand keywords! We are talking about big businesses that have #1 organic ranking for their well-known brands. It is very interesting (to be said politically correctly), because the the study referred here was published back in 2013 and had been discussed in professional media, it isn’t something totally new.

Have all of them conducted experiments similar with eBay and came to different results? To be honest I doubt, it looks more like good job of sales and account managers of agencies and Google and inability to make experiments and learn from them. As Professor Tadelis said “A lot of businesses do not have a culture of learning. Someone thinks they know what’s going on, and no-one challenges them”. It’s not easy to challenge yourself and admit that something may be wrong with existing practices.


  1. Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment
  2. Culture of learning key for data success , notes from ‘Talking Data’  event with Prof Steve Tadelis, Melbourne Business School website

Leave a Reply

Your email address will not be published. Required fields are marked *