An intervention is only as good as the evidence it's based on: the case of Nudge supermarket

The popularity of behavioural science means that it sometimes gets used as kind of window dressing or "fairy dust" to make public initiatives seem better while not addressing the root cause of the issue or not solving the right problem due to evidence that is not robust enough - or because we are biased as practitioners. In this article, I dissect an example of "nudging for good" that looks good on the surface but on deeper investigation reveals issues with the premise, the data and the recommendations - and offer some improvement suggestions.

This morning I spotted a tweet about the launch of the UK’s first supermarket called “Nudge”, designed by public health experts, that aims to "showcase how the retail layout and shopper experience can be transformed to encourage customers to make healthier choices". It appears to be a follow-up project from a report a recent report released by the Royal Society for Public Health (RSPH) and Slimming World called ‘Health on the Shelf’ that calls for supermarkets to join the fight against obesity – a major public health issue in the UK.

Sounds like a great example of nudging for good? I’m not convinced so let’s dig deeper.

Issue 1: The premise

“If supermarkets empowered their consumers to make these changes themselves though through creating an environment which promoted a healthier diet, they could become part of the solution in helping tackle the obesity epidemic.” (Slimming World Public Health Manager and Dietician Carolyn Pallister)

While supermarkets undoubtedly influence choices, the bigger problems remain the same: level of poverty in UK & low prices of poor quality staples means obesity is largely a class issue (here and here). This means that rearranging supermarkets, especially on based in one of the least affected areas in the UK, Borough of Islington, will probably not make much of a difference.

As one research paper in Social Science and Medicine remarks, "food pleasure is affordable, accessible, immediate and reliable in a way that other pleasures are not".

In other words, for those who have very little by means of resources, food might be one of the few pleasures they can afford - and making (un)healthy food a moral issue for individuals is a good example of the empathy gap between those who have power and those who do not. I'm fairly sure that everyone involved in this report and the implementation of its recommendations is of a certain social class by virtue of their education and income levels, and as such the class privilege when it comes to food is less salient to them. Likewise, many/most behavioural scientists are middle class or above and as such they, too, are biased to think about both obesity and healthy eating from a particularly individualistic perspective.

Why is this an issue? Well, if we want to design interventions that truly work, we need to genuinely, sincerely understand behaviours that are "problematic" (by someone's definition) and the context they occur in. Being poor also limits your mental bandwidth - i.e. "brainpower that would otherwise go to less pressing concerns, planning ahead and problem-solving". Resource scarcity can lead to "a life absorbed by preoccupations that impose ongoing cognitive deficits and reinforce self-defeating actions" - including making unhealthy food choices.

A related class issue is that over 7 million consumers in the UK are functionally illiterate(also here) and as such unable to educate themselves about healthy eating. Both problems would require extensive reform in UK society so shifting the responsibility to individuals and supermarkets is a convenient way to avoid addressing the much bigger underlying issues. As Martin Cohen wrote in the Independent in 2018:

Obesity is invariably presented as a diet issue for nutritionists, whereas social inequality is deemed the domain of sociologists and economists. Put another way, even as the inequality gap becomes more and more obvious there’s been a medicalisation of a social problem. Yet obesity is not just a matter for nutritionists: rather, it is a product of social inequality and requires a collective social response.

Issue 2: the data

Moving on from the premise, let's take a look at the data this report is based on.

"A panel of experts in public health, nutrition, diet, weight management, consumer insights and retail, were invited to discuss what a healthy supermarket could look like and how supermarkets could be more effective in nudging people towards healthier behaviour. -- The panel’s recommendations were supported by public polling of 2,084 adults in the UK, and 2,000 Slimming World members, to understand their views on how supermarkets are contributing to the obesity epidemic and how they could do more to help people live healthier lives."

For start, the sample is partly composed of Slimming World members, and there is no information on how the data might have been weighted to correct for this self-selected group of people.

Secondly, the purpose of the research appears to be to understand very specifically how (not if) supermarkets are contributing to the obesity epidemic and how supermarkets could do more to help people live healthier lives. This goal would explain some of the examples of data that have been made public:

"Fewer than 1 in 5 (15%) of the public believe supermarkets are doing enough to tackle obesity"
"Almost half of the public (48%) believe that legislation should be introduced to require supermarkets to proactively encourage healthier choices"
15% of people claim supermarkets make them go off track when it comes to healthy eating

As an applied behavioural scientist, I would be very cautious about taking statements/claims like these at face value - if the methodology simply consisted of asking people to endorse statements like these, there is a huge risk of response bias. Without knowing more, I am sceptical of how much credibility should be attached to self-reports like these where there is a clear opportunity for motivated reasoning and responding in a way that most effectively protects the survey respondent's identity.

Issue 3: The recommendations

Let's take this section by section.

I can't claim to have superior inside knowledge of how a supermarket works, but I can't help but feel that this first section of recommendations are lacking a dose of reality - the reality that supermarkets are highly complex commercial entities where even a change that seems small to the outsider requires hundreds of hours work, let alone anything else. Therefore, even just one of these recommendations would be a huge operation by itself and some, like the last one, are logistically near impossible due to health and safety regulations for storing food.

The second group of recommendations is equally problematic and makes a number of assumptions. Just to name a few:

What does empowering signage mean in reality?
Handing out healthy snacks and free food - who bears the cost for this?
"Knowledgeable, friendly staff with a community feel" - it's a supermarket, not a social club, and again the responsibility is shifted away from society and on to individuals

Again, picking out a couple:

renaming or rebranding healthy products - the why and how are missing, not to mention who bears the cost of these rebranding exercises
live demonstrations - not to sound like a broken record, but again, who pays?

A lot of these recommendations seem out of touch with the reality of running a major supermarket operation and the potentially enormous costs involved in implementing any of them. Unfortunately, the lack of actionable recommendations renders the initiative as a type of virtue signalling.

Key issues - a summary

Although the biggest problem with this initiative is that it does not address the likely root cause of obesity due to (presumably) political reasons, it has a number of smaller issues too.

Impact not measured: Although it is possible Royal Society for Public Health (RSPH) and Slimming World are measuring the impact on behaviour of this pop up store, neither the report nor the news articles mention it. If they are, what and where is the control store? If there is no measurement of the impact of an intervention, it is merely window dressing - a behavioural science party trick aimed at making someone look good.
The experiment is not conducted with the right target audience: The shoppers who are likely to come to this pop up store are not the key target audience – they are more likely to be mostly middle class and relatively speaking pretty well off. Even if there is a measured and significant impact, there is no way of being certain that it would work with the people that are in most need of help.
Lack of scientific frameworks and holistic, empathetic situation analysis: using a framework such as COM-B and the Behaviour Change Wheel from UCL Centre for Behaviour Change would be a good start for understanding the complex and nuanced context of people's behaviour and linking it to tried and tested types of interventions.

The bottom line?

As a professional who specialises in applying behavioural science, I have a keen interest in the reputation of our "industry". Although it is great that more and more people want to use behavioural insights, there is a danger of diluting its image when interventions are not successful - a danger that is more likely if the intervention is not based on robust evidence because simply sprinkling some "behavioural fairy dust" on projects is not enough.

A more fundamental challenge is that we often do not see our own privilege and that we are solving problems for other people from that unique perspective. Great examples of this can be found, for example, in the Empathy Delusion report by Reach Solutions and House51 - although their report focuses on the advertising and media industry, I believe their findings also apply to those working in behavioural science, applied or otherwise.

The power of any behavioural intervention lies in having an accurate diagnosis of the issue, robust evidence of the factors influencing it as well as our ability to look at the issue from multiple angles - not just the one that is the easiest or most comfortable to us.

First published on LinkedIn 1st August 2019 - lots of lively discussion in the comments!