Using acupuncture as an anaesthetic during surgery. Image originally uploaded here
I spent some time yesterday talking to a friend that is studying Chinese Medicine.
She is training to be an acupuncturist and she is interested in the different research methodologies used in evidence based medicine.
This is a big issue for complementary therapies in general as they believe in using qualitative as well as quantitative research to understand a treatment’s efficacy since most complementary health treats the whole person and not just observable symptoms.
And it is a big issue for acupuncture in particular as the standard means to test any medical treatment, the randomised controlled trial (RCT) is notoriously problematic for this therapy. You can undertake a RCT but it involves using sham needles on the control and so while a single-blind test is therefore possible the gold standard of a double-blind test is out of the question.
She wanted my help in understanding different research methodologies.
But as a planner working in marketing communications I realised I could be of little help since I rarely come across research. Or rather I rarely come across real research and our little discussion finally proved this to me.
What we spend most of our time dabbling in is the application of once credible research techniques in wholly inappropriate ways to tasks for which it is ill suited. Indeed, that the sham research in which we engage goes by the name of research is one of the greater crimes against the English language.
Let me be a little more specific.
There appear to me to be three types of ‘research’ that we use in the marketing business, and only one should carry the title.
When we are using quantified data that reports actual behaviour by consumers I think we can call that research. Whether it is derived from the behaviour of the universe (e.g. a brand’s volume share) or from a sample of the universe (a brand’s penetration) it tells us about something that has actually happened in the real world. And when we test something in the market and report on it’s success I think we can call that research too – a regional launch, split copy testing, and classic direct marketing testing. This is all data you can depend on and make decisions with.
What we call qualitative research is also equally useful, when it is used appropriately. But when we use it appropriately I’m not sure that it deserves the name research either. Qualitative research works best in our business as ‘food for thought’ and as a powerful source for creating insight. Qualitative research, doesn’t tell us what is going on, at best it gives us some ideas about what might be happening. And it does this in a way that may help to trigger a new way to define or crack a problem in the planner’s mind but is laughable as a means for making critical business decisions.
I had a client recently that refused to call ‘creative development research’, research at all and maybe we should follow that lead. How about calling it qualitative exploration from now on, after all that is what we are doing it for – to have a little subjective, unaccountable rummage in pursuit of a little nugget that we can use to help understand what people think and why and what might change their minds.
Both real research and qualitative exploration are legitimate activities.
What passes for ‘quant’ nine times out of ten in this business is the bastard, as sham as those needles used on acupuncture control groups.
Whether it is a Usage and Attitudes survey, Pre-testing or Tracking these activities offer the illusion of respectability and certainty by borrowing the clothes of the kind of research that gets drugs approved for launch but represent little more that quantitative speculation.
The problem for me is part philosophical and part practical.
Philosophically the idea of quantifying claimed behaviour emotions and attitudes seems a most peculiar activity. It may be interesting in and of itself but it gives us very little idea of the behavioural implications. Medical researchers don’t ask people whether they think that the drug they have just tried is likely to work on a five point scale. They find out whether it did work using proper research.
And practically, as the medical profession knows all too well, undertaking proper quantitative research is a expensive and time consuming and the two things marketers are strapped for right now are cash and time. How many times have you given or sat in a debrief where the cell sample sizes you are seeing precentaged are lower than 100? How often is it that the quotas for the samples have not been met or are unbalanced because the researcher agreed to turn around times that are way too tight.
Do this kind off stuff by all means – after all we need some help to make decisions as we negotiate our way through the complex and risky process of building effective marketing communications. But lets stop calling it quantitative research.
So from now on I am going to reserve the name research for the real deal – actual data that reports on reality.
I am going to continue my affection for great qualitative exploration, it is not the only or often best source of insight but sometimes it is dynamite.
But when it comes to the other stuff, I’m going to call it quantitative speculation, I invite you to join me.