18 Comments
Jul 25, 2022Liked by Stuart Ritchie

I'm sorry to bother you with this as I am sure you must be thinking about it, but what do you think of the recent review which claimed that serotonin has no relationship to depression? Would love to read your take on it.

Expand full comment
Jul 26, 2022Liked by Stuart Ritchie

(I sincerely hope) there’s a special place in hell for government bureaucrats and nudge-ists who think “changing the default” is a benign “nudge”.

Making something opt-out rather than opt-in is not some brilliant science-based choice architecture hack.

It’s simply coercing users by taking away their freedom to choose.

Compare “changing the default” to other judges and it’s clear that this is the most coercive technique (yet).

It is therefore no surprise that this is mentioned as one of the more effective methods.

Sorry for the rant but I had to get this off my chest.

Thank you for this essay/post (sorry, no idea what substack posts should be called), I really enjoy your writing

Expand full comment

One major assumption here that is fundamentally wrong in the analysis of publication bias, is the requirement that all the reported effect "clouds" must be symmetrical even *throughout the zero effect*. Let me demonstrate with an example. You design a study to see whether say.. injecting bleach can cure Covid-19. You gather 100 people, inject 50 with bleach, 50 with saline. Hmm, weird, you replicate the study. Ten times, hundred times. You can plot funnels, clouds, but even if by some miracle you see a dot on the right, expecting any kind of symmetry is detached from reality.

So, yes, the variability in experimental design plays a role, but that's not all. You need to account for the fact that these experiments have been designed with a specific goal to achieve positive effects. And that may be a faulty mindset in science generally, but it is the fundamental nature of what nudges are about! In some cases, observing a negative effect size is actually absurd or even impossible. If you observe how many people sign up for organ donation form that's innocuously bundled to a driver's license application - would you expect that someone flips the table, storms off, and writes a last will saying: "When I die, dissolve my body in a vat of acid." ?

And even though thorough statistical interpretations are very much needed, sometimes the world is too complex for us to reduce into numbers.

Expand full comment

i think a better term than "nudge" would be lying, manipulation, coercion, control etc. "nudge" is a propaganda term used to minimize the reality of how evil and sociopathic "nudging" (manipulating) people really is.

Expand full comment

This does seem to come up often. I wonder if we should train ourselves to think "Meta-analysis - Is it publication bias?"

Expand full comment

There's something I don't understand about the treatment here. Aren't all nudges apples and oranges, even closely related ones of the same type?

If one study finds with high certainty that a wasp picture in urinals have a positive effect and another study finds with high certainty that a butterfly picture in urinals have a negative effect (of the same size), is the relevant meta-conclusion that urinal-picture-nudges have no effect on average? Wouldn't the relevant conclusion for real-world practical applications be that nudges can be effective, but you have to test the effectiveness of the given nudge because you can't presume to know what it'll be in advance?

How does it make any kind of sense to look at an "average effect" of studies that study different nudges (even of the same type) and conclude that merely because the average is zero or very small - or would be if not for publication bias - this means we have no evidence for effectiveness of nudges in that domain overall? Just because a randomly designed nudge will have little to no effect on average doesn't mean nudges can't be effectively employed with the proper methodology for designing and testing them. Testing just has to be a part of the deployment process itself, rather than something left for academics to do later. or do I misunderstand something here?

Expand full comment

The problem with having a nudge department is that, no matter what the situation, they're never going to say 'let's just be honest with the public and lay out the facts before them.'

As a nudger, your job security depends on persuading your bosses that the general public are a bunch of suicidal buffoons. Which can't be good for society.

Expand full comment

You are British and the "nudge" guys are American yeah? I saw you say "pension scheme" and I knew you had to be British because in American English "scheme" is incredibly negative in connotation whereas "nudge" is about as neutral as you can get for what it is describing. We'd usually call them "pension plans" rather than "pension schemes". Planners are good and schemers are bad.

Expand full comment

Many thanks Stuart for this excellent article. It would make a great addition to the nudge theory articles series I've been publishing at https://realkm.com/nudge-theory/

I wonder if I could obtain your permission for republication? The article would be republished verbatim, and I'm very happy to add backlinks and attribution as required.

I look forward to hearing from you, and can be contacted through https://realkm.com/contact/

Many thanks,

Bruce Boyes.

Expand full comment

This may be overly optimistic, but perhaps we can take encouragement that the open data provided alongside the original meta-analysis must surely be a factor which facilitated the speed of the corrections and commentary?

Expand full comment