Replications will not help: The paradoxes motivating Prospect Theory still don't tell us much about human risk-preferences

Like Comment

This post was co-authored with Dr. Håkan Nilsson, and Prof. Peter Juslin (Department of Psychology, Uppsala University). 

A recent study in Nature Human Behavior provides a replication attempt of Kahneman and Tversky’s canonical 1979 study which motivated Prospect Theory (1). The authors present a textbook example of how to conduct a stringent replication (or any study involving large-scale data collection for that matter). Unfortunately, the authors draw unwarranted conclusions, stating that their findings indicate that the theoretical constructs underpinning Prospect Theory  are reliable, and that researchers are well-off using Prospect Theory as a viable explanation of individual behavior. We argue that research not cited in the article offer strong evidence against these conclusions, and that their results are actually in line with this conflicting evidence.

There is a long tradition in the behavioral sciences to use gambling tasks involving monetary lotteries to capture revealed risk-preferences (RRPs, which stands in contrast to stated risk-preferences common in verbal self-reports; e.g., “do you consider yourself to be risk averse?”). Recent years has seen an intense effort to uncover the convergent validity and temporal stability of RRPs, and map how they relate to other psychological constructs, such as cognitive ability (e.g., numeracy, the ability to understand and operate on numerical information) and personality (e.g., Big-5, impulsiveness). Large-scale studies have found that RRPs elicited from monetary lotteries fail to capture stable risk preferences (2, 3). Instead, it seems that RRPs, to a large extent, define relatively advanced demands on people’s ability to process and understand numbers and abstractions such as “probability”, rather than risk preferences per se  (4, 5). The results of Ruggeri et al. is indeed in line with such a conclusion; the patterns replicate the strongest in the countries with the highest levels of education. It will not matter how much the paradoxes replicate if the experimental stimuli cannot be validly used to elicit individuals’ risk preferences: a theory about risk preferences need to be grounded in tasks that indeed capture risk preferences.

Of course, Prospect Theory can still provide a quantitative account of the data, and has, in a variety of research fields, indeed been successful in describing empirical phenomena correctly. However, Prospect Theory (and its successor Cumulative Prospect Theory) intends to be a homeomorphic model, meaning the aim is to give the parameters of the model plausible psychologically interpretations that capture the underlying processes (6). That Prospect Theory often fails to capture the relevant psychological mechanisms is painfully evident looking at the intra-individual data: typically, people do not exhibit much more than half of the expected paradoxes. Looking at inter-individual variation tend indeed to delude findings at the intra-individual level (7). Recent simulation-driven research has actually shown that one can produce the results by Kahneman and Tversky (and hence Ruggeri et al.) when assuming different individual-level mechanisms than those proposed by Prospect Theory (8).

In sum, empirical findings need to not only fully replicable but also fully understood. Otherwise, some effects may still linger as urban myths conveniently used to explain phenomena on a post-hoc basis, while the true psychological explanation remains unclear. A unified descriptive theory of decision making under risk and uncertainty need to account for individual differences, and although Prospect Theory fail in this regard there are other viable models that offer initial promise (9,10).

References:

  1. Ruggeri, K., Alí, S., Berge, M. L., Bertoldo, G., Bjørndal, L. D., Cortijos-Bernabeu, A., ... & Gibson, S. P. (2020). Replicating patterns of prospect theory for decision under risk. Nature Human Behaviour, 1-12.
  2. Frey, R., Pedroni, A., Mata, R., Rieskamp, J., & Hertwig, R. (2017). Risk preference shares the psychometric structure of major psychological traits. Science Advances3(10), e1701381
  3. Pedroni, A., Frey, R., Bruhin, A., Dutilh, G., Hertwig, R., & Rieskamp, J. (2017). The risk elicitation puzzle. Nature Human Behaviour1(11), 803.
  4. Lilleholt, L. (2019). Cognitive ability and risk aversion: A systematic review and meta analysis. Judgment & Decision Making14(3).
  5. Cokely, E. T., Feltz, A., Ghazal, S., Allan, J. N., Petrova, D., & Garcia-Retamero, R. (2018). Decision making skill: From intelligence to numeracy and expertise. Cambridge handbook of expertise and expert performance. New York: Cambridge University Press.
  6. Wakker, P. P. (2010). Prospect theory: For risk and ambiguity. Cambridge university press.
  7. Kirman, A. P. (1992). Whom or what does the representative individual represent?. The Journal of Economic Perspectives, 6(2), 117-136.
  8. Regenwetter, M., & Robinson, M. M. (2017). The construct–behavior gap in behavioral decision research: A challenge beyond replicability. Psychological Review, 124(5), 533.
  9. Broniatowski, D. A., & Reyna, V. F. (2018). A formal model of fuzzy-trace theory: Variations on framing effects and the Allais Paradox. Decision5(4), 205.
  10. Mishra, S., Barclay, P., & Sparks, A. (2017). The relative state model: Integrating need-based and ability-based pathways to risk-taking. Personality and Social Psychology Review21(2), 176-198.

No comments yet.