Error culture in science in the face of a global crisis

Given the need to move ahead quickly and generate evidence, there is always the danger of error. The question is, how to best deal with this as scientists.

Like Comment

Niklas Keller, Charité University Medicine & Simply Rational Ltd
Stefanie Marx, Goethe University Frankfurt
Sabrina Artinger, German Federal Chancellery & Max Planck Institute for Human Development
Florian M. Artinger, Berlin International University of Applied Sciences & Simply Rational Ltd & Max Planck Institute for Human Development


At a press conference in mid-March, 2020, Dr. Michael J. Ryan, Executive Director of the World Health Organization (WHO) Health Emergencies Programme, shared what he considers essential to combatting the emerging COVID-19 pandemic: “Be fast. Have no regrets. You must be the first mover. The virus will always get you if you don’t move quickly. […] Speed trumps perfection. And the problem in society we have at the moment is everyone is afraid of making a mistake. Everyone is afraid of the consequence of error. But the greatest error is not to move.” For many urgent questions that concern the pandemic, there are no ready answers.1–3  Science itself has gone into rapid response mode to quickly generate evidence and publish COVID-19-related material.4 This is not how science usually operates, heightening the potential for errors, even as public policy makers continue to rely on scientific advice.

A case in point is the ibuprofen scare that also arose in mid-March: a correspondence letter published in Lancet Respiratory Medicine postulated a link between ibuprofen intake and worsening COVID-19 progression.5 The arguments on which this hypothesis rested lacked a sound mechanistic basis and clinical evidence. Yet, the letter was nonetheless published and the issue snowballed into a global health scare, during which the WHO and many national healthcare bodies initially advised against the use of ibuprofen, recommending paracetamol instead. Although these recommendations were withdrawn within a week, people rushed to stock up on paracetamol, causing shortages in hospitals where it is an important and widely used medication in intensive care.6

Rapid response implies that policy makers must often rely on heuristics, which can be effective tools when information is scarce but cannot guarantee optimal outcomes.7,8 In dealing with the current crisis, for example, policy makers have been relying more than ever on the “trust the expert” heuristic and scientific prestige. As the case of ibuprofen shows, this approach is imperfect and can lead to errors particularly if science itself operates in a rapid response mode. There are two approaches to dealing with errors9: In a negative error culture, errors must be avoided at any cost, requiring enormous resources and slowing or even halting progress. Solutions that are in the interest of the public but involve some risk are not explored. Instead, decision makers opt for an inferior solution that allows them to “cover their ass” in case something goes wrong.10 In a positive error culture, mistakes are explicitly acknowledged as an inevitable and necessary part of the learning process in an uncertain environment. If there is an unexpected negative outcome, the primary goal is not to identify a scapegoat but to identify underlying systemic causes. This ensures effective and speedy learning, ideal for providing the best evidence possible while maintaining the ability to act.

In uncertain times, our task as scientists is not only to deliver evidence. We must also educate policy makers and the public on how best to deal with error and uncertainty.

 

Bibliography

1.        van Bavel, J. J. et al. Nat. Hum. Behav. 1–12 (2020). doi:10.1038/s41562-020-0884-z

2.        Chater, N. Nat. Hum. Behav. 1–1 (2020). doi:10.1038/s41562-020-0865-2

3.        Betsch, C. Nature Human Behaviour 1–1 (2020). doi:10.1038/s41562-020-0866-1

4.        Editorial Nature Human Behaviour. Nat. Hum. Behav. 4, 327–328 (2020). https://doi.org/10.1038/s41562...

5.        Fang, L., Karakiulakis, G. & Roth, M. The Lancet Respiratory Medicine 8, e21 (2020).

6.        McKenna, M. WIRED (2020). https://www.wired.com/story/th...

7.        Tversky, A. & Kahneman, D. Science. 185, 1124–31 (1974).

8.        Gigerenzer, G., Todd, P. M. & the ABC Research Group. (Oxford University Press, 1999). doi:10.1002/mar.10060

9.        van Dyck, C. et al. J. Appl. Psychol. 90, 1228–1240 (2005).

10.      Artinger, F. M., Artinger, S. & Gigerenzer, G. Bus. Res. 12, 9–25 (2019).

Go to the profile of Florian Artinger

Florian Artinger

Professor, Berlin International University of Applied Scienes

No comments yet.