Why focus groups aren’t enough
‘What influenced your decision to study at your chosen university?’
Ask a bunch of students and you’ll hear them recount personal stories about why they chose to study at their university: it was the interesting course content, the reputation of the university, or the good feeling they got at the Open Day.
University marketing teams often conduct focus groups and surveys as we seek to understand what drives students’ decision-making. We use these insights to inform the content and campaigns we create. So far, so normal.
But research suggests these decision-making stories may be just that, stories. Our judgments can be influenced by a messy set of factors beyond our conscious awareness. And while we can tell a compelling narrative about why we do things, we often have poor insight into what actually drives us.
What we say isn’t what we do
A study conducted at UCLA looked at how students’ responses to persuasive messaging would predict their behaviour. Students had their brains scanned while they looked at public health messages about sunscreen, a suitable topic for an institution located in sunny Los Angeles. After the scan, participants were asked whether they intended to wear sunscreen in the coming week. Their responses varied; some declared they would start using it every day, and others confirmed they planned to continue their sunscreen-free ways. A follow-up a week later revealed what they had done.
‘The relationship between what people said they would do and what they actually did was negligible.’ Matthew D Lieberman, researcher
That’s right, people’s actual behaviour bore little relation to what they said they were going to do. But interestingly the brain activity in the scanner did predict their activity. Specifically, the more participants' medial prefrontal cortex was activated by the persuasive messages, the more those individuals increased their sunscreen usage later on, regardless of what they said they intended to do.
This shows not only that intentions don’t predict behaviour but also that in stating our intentions, we don’t have insight into the present activity in our brains which could predict our behaviour. We lack introspection.
What does this mean for market research?
This lack of introspection means that asking people why they prefer something or what influenced them can be misleading. So what can we do to gain insight?
Well, we can’t easily put people in brain scanners. Here are some alternative options:
Look for more information.
It’s easy to get hooked on a good explanation. For example, if your prospective students tell you they chose your university because of the buzz at Open Day, then chances are you will accept that explanation and busy yourself with making the next Open Day as good as possible.
This is an example of what Daniel Kahneman calls ‘fast’ thinking. And the error here is not only your Open Day attendees’ potential lack of introspection but your quick acceptance of their explanation. Kahneman warns us that our confidence in causal explanations is determined by the coherence of the story, rather than the quality and amount of information that supports it. That is, there may be a huge multitude of complex, messy factors which influenced a particular person’s decision but we zone in on the story which makes sense.
The way to navigate around this flaw in our thinking is to look for more data and seek out information which may contradict the story, so we can get a better understanding of the complexity.
For market research, this means not taking a single account at face value but triangulating the data. This leads to our next point…
Look at the behaviour, not the intention.
In searching for further information, we should look not just at what people said but what they did.
Thinking back to the sunscreen study, it would be easy to conclude which message was most effective based on people’s stated intentions. But it’s important to look at how they actually behaved to understand the impact.
Your web analytics, sign-ups and application data will tell you how people behaved. For example, you may find that people who attended an Open Day were indeed more likely to enrol. While this shows a relationship between these two things, it doesn’t necessarily show that Open Day attendance caused people to enrol. Were they more likely to attend the Open Day because they were already more interested in enrolling in the first place?
Ultimately, we are looking for predictive factors. And to do this, we need to test things.
Test things
To really understand what factors influence behaviour we need to take a scientific approach and test our thinking. To do this we need to:
Observe phenomenon
Develop a hypothesis
Formulate a prediction
Test the prediction
Iterate
For example, based on our observations we may come up with the idea that personalised communications will lead to more applicants accepting their offer than a generic alternative. To test this, we’d need to split applicants into two comparable groups: one group receives the standard communications, and the other receives the personalised version. We’d then track the conversion of these two groups to identify if there was any difference in their behaviour.
I have seen examples of university teams carrying out tests exactly like this and gaining much stronger insights as a result. But I’ve also seen teams hesitate and not take this approach. This may be due to lack of time, perceived effort or even concern about offering different experiences to different groups. But this seems like a missed opportunity. If the scientific approach is good enough for life-saving medicine, isn’t it good enough for us?
We spend a lot of time creating content so it is important to understand if what we deliver has a positive impact on our audiences. Ironically, research suggests that some of our traditional market research methods are not sufficient so we need to dig deep to truly understand the impact of our content.
We can help.
Pickle Jar can help you turn insights into strategy. If you have current research which isn’t performing, have a rough idea of what insights you would like to gain, or have no idea at all. We’re here to help you, get in touch with us at hello@picklejarcommunications.com
References
Matthew D. Lieberman, Social: why our brains are wired to connect
Daniel Kahneman, Thinking fast and slow
