One of the first stories you hear when entering the advertising industry is John Wanamaker‘s famous quote – which has been bent over time but is a variation of
“I know that half my advertising spend is wasted
I just don’t know which half.”
Clearly this wasn’t good enough for John, and certainly isn’t the ideal scenario when the amount wasted can represent millions of pounds so an entire sub-industry has grown in advertising which is that of measuring ad effectiveness.
At a basic level ad effectiveness can me measured in 3 ways:
Direct measurement of sales:
In Direct Response marketing there are various methods of attributing sales to the media that triggered it. In online the bluntest way is the “last click wins” scenario, but there are also tried and tested methods for other media – dedicated phone numbers that will differ by medium or by copy iteration; coupons with codes to be quoted during a phone or store visit. Each has its values but as a general rule quoting coupon codes or source when visiting a store, buying on the phone or online is afflicted with a “tick the first one” mentality. List orders are changed but still there will be errors.
Measuring actions implying purchase intent:
This could be online enquiry forms filled out, store locator searches; phone calls, brochures sent out, appointments booked, actual store footfall – all of which are fairly taken as an indicator that the customer is getting closer to opening their wallet. Again there can be issues as it may be an indicator, but not a true reflection of what happens later.
This involves researching potential and actual customers, and asking them what they think. Often there will be a “brand recall” metric – how many people remember the name (prompted or unprompted); a “purchase intent” question and other factors that have been proven over time to reflect consumers’ eventual actions and indicate that a campaign has had a positive impact – recognition of ad straplines/ association with intended characteristics such as “trustworthy” or “good value”. When used with a large enough sample size to ensure statistical validity.
One of the many reasons that digital advertising has grown is its very measurability. Being able to see how many times an ad has been seen (impressions), its response rate (clicks) and the resulting actions (sales, enquiries, online brand research) means that in near-real time an advertiser can tweak the campaign to ensure that creative or ad copy that performs the best is the one that is seen most, and that they get the most out of their ad spend.
But do they?
The problems start with the default model for online ad measurement, which is last-click-wins. There are various factors that undermine the validity of this model, and these are best understood by remembering that the digital world is just one part of any consumer’s methods of interacting with a brand, and even that digital, trackable world is fractured.
The first is the basic assumption that the last thing someone did before a purchase is the most important. In the image below you can see a simple example of how someone researching and buying online interacted with different ad events:
In this instance the brand search would get all the credit for the conversion, but we can see that they interacted with the advertisers messaging in 3 other ways before then – all of which may or may not have had an influence on their later behaviour. A similar metaphor is the football team – although a striker may have scored the actual goal, the rest of the team are crucial in setting it up, defending their own goal etc.
If you planned your advertising spend based on the last click model the risk is that your ads wouldn’t be showing in the places that do help the process along, and therefore would sell less. In recognition of this the industry has spent the last few years discussing methods of sharing out the credit to “attribute” influence more fairly and have a more effective media strategy.
Some of these models are laid out below, showing from the top
- “first and last” – where an advertiser believes (or the data shows) that the first and the last interaction carry the highest weight, the sale value is shared amongst them
- “all but last click” could apply when the last click is proven to be a navigational brand search, and the credit for influencing the sale is shared amongst the interactions that had a harder job of persuasion
- “shared” is where all the events get equal weighting as it’s proven that the sale relies just as much on each element
- “last click” is the traditional model that is blunt but still the fastest data to access
Well, yes and no.
Firstly to work out the one that works best it requires an enormous amount of data crunching – the first days of Exposure to Conversion (or E2C – Doubleclick’s version of attribution modelling) required a data dump that broke excel. Just the discussions of which model to use can take days of discussion and each time you’d settle on a model there’d always be a “It just doesn’t feel right” comment from one stakeholder or another, and you’d have to re-assess the whole thing with a different cookie window. Try doing that every day and making hundreds of creative and channel optimisation decisions on the fly.
Secondly, it’s still only a partial view of the data. Remember that one person in this instance actually means “one device with a cookie on it”, and the last two years in particular have broken the desktop stranglehold for good; add to this the fact that most sales are still conducted in retail stores or on the phone, and you have a multi device and cross channel world:
Even if you can match a user across multiple digital devices (which you often can’t), tracking that same person as they watch telly, pick up the phone or walk into a shop adds complexity that reduces your chance of getting robust enough data in enough time to be actionable during the advertising campaign.
So what to do?
Firstly, realise that you’re not going to get a perfect answer. The beauty of digital is often seen as its trackability although this has lulled us all into a false sense of security about its accuracy; in the real world people always have been influenced by more things than you realise, so you have to cast the net as widely as possible. To see the bigger picture and treat the consumer as just that, as opposed to a cookied device, you really need to jump into the realm of econometrics, which research companies and large agencies have used for decades to apply some kind of cause/effect on their advertising activities and assess the true impact of each part of their marketing activity.
Put simply econometrics takes a vast amount of data and tries to find correlations. For the marketeer it can show the impact their media has on their sales (or any other known metric); how store location impacts success; and how weather/external market factors influence the overall base line of sales that you could expect without any other stimulus. The results from econometrics for a marketeer are often a) a sand diagram showing the influencing factors over time, and b) a Cost Per Acquisition hierarchy showing the difference between the linear (last click) tracked CPA, and the one inferred from the econometric models.
What often happens is that media that occurs in the earlier stages of a research/buy process proves to have been more influential than could be seen from the usual attribution model, which gives them credit for more sales. The difference between the linear sales and the modelled sales becomes a “multiplier” which is then applied to the linear results to show the true value of each media, and to aid in effective budget allocation.
The solution, ish.
My approach is always to use all the data I get get my hands on, in the most appropriate way.
Planning annual/quarterly media budgets needs the input of econometric multipliers to ensure that the stimulus media that kick off the consideration process get enough juice to start the pipeline off. Handily this kind of data is also a great help to show the FD that there is in fact a value in the media that otherwise struggles to prove its worth.
All this is well and good, but I have never met a marketeer or planner, brand or direct who doesn’t have to justify their weekly or monthly plans based on last click data, or at best a “last click plus “assists” model, so we have to face facts that the measurability of online is also a stick with which it can be beaten. For great context, look no further than this quote (often erroneously attributed to Einstein, but actually the work of sociologist William Bruce Cameron.)
Not everything that can be counted counts,
and not everything that counts can be counted.
In other words, we’re human.