How we learn to value the road not taken: Deliberation and Memory shape the valuation of unchosen options

Like Comment
Read the paper

Imagine you are at a new restaurant, deliberating between a salad and pasta. You choose the pasta, taste it, and are disappointed to learn it isn’t as good as you expected it to be. Prominent models of decision-making predict that you will devalue the pasta and are not likely to choose it next time. Indeed, a growing body of work describes how we learn from the outcomes of our choices and update value accordingly1.

Yet, little is known about the fate of our unchosen options: the salad, in the example above. In our recent Nature Communications paper, we discovered that the value of unchosen options is also affected by the outcome of our choices, despite the fact that we never really had the chance to learn about whether that option was good or bad. Specifically, we found that learning the value of the chosen option changes the value of the unchosen option, but in the opposite direction. If the pasta you ordered was disappointing, the salad might become more desirable the next time around – even though you never actually tasted it.

Why would learning about the pasta affect the value of the salad? It comes down to a process that starts during the initial deliberation itself. The goal of deliberation is to contrast the options, so that we can commit to one and forget the other. Yet, counterintuitively, deliberation also ties the two options together, creating a lasting association between them. We hypothesized that this association can have later consequences, such that learning about the chosen option brings to mind the associated unchosen option, leading to changes in the value of both of them.

To test this hypothesis, we designed a set of online behavioral experiments (using Amazon Mechanical Turk). We asked people to act as art auctioneers. We showed them pairs of paintings – say, a house on a river and a man with a hat – and told them to choose which one to sell in an upcoming auction. Then, we revealed how well their selected pieces did at the auction. For example, the “man with a hat” painting might net $150 or nothing. Participants knew their decisions mattered: they were told that for one of the paintings, they would actually gain a percentage of its net worth. Importantly, we never revealed anything about the paintings they did not choose.

We then presented a new set of decisions. This time, we showed people new combinations of paintings -  pairs of previously chosen paintings, and pairs of paintings they had passed over. For chosen pairs, we expected people to select pieces that did well in the auction and that is exactly what we found.

But how would they decide between unchosen pieces they had no knowledge about? We found that when deciding between unchosen paintings, people preferred paintings that were previously paired with pieces that did not do well. We call this effect an “inverse decision bias”.

To find out whether the inverse decision bias was related to the deliberation itself, we tested people’s memory for the pairs of paintings they saw at the very beginning. We found that the extent of the inverse decision bias was strongly predicted by how well people remembered the pairs they deliberated about. This finding highlights the interplay between memory and decision making. It suggests that the (not so) simple act of making a decision leaves a memory trace, in the form of a link between the two options. When people try to disentangle the values of different options, deliberation itself entangles the options in memory, with lasting consequences for future choices.

We replicated our main findings across five distinct data sets (612 people participated in our task in total), and our main experiment (including 235 people) was pre-registered on Open Science Framework. All data and analysis codes to reproduce our findings are publicly available2.

You might ask yourself why the bias is in the inverse direction. Why don’t participants infer that if the pasta wasn’t tasty, the salad wouldn’t be either? Indeed, previous research has shown that value can be directly transferred between random objects that became associated3. What makes our study different is the process by which the association is formed. We ask people to make a decision and deliberate about it. To do so, people compare the options and create a contrast between them. Our work suggests that this comparative nature stays after the decision is made and may be the basis for the inverse association between choice options.

The idea that people continue to contrast choice options resonates with previous studies. For example, studies that assess memory biases following choices4–6, have found that people tend to better remember and learn the positive aspects of their chosen options and the negative aspects of their unchosen options, thus increasing the contrast of value between the options. A similar tendency is seen in studies that assess changes in preferences as a function of choices7–9. The typical finding is that after making a choice – even without any feedback on the choice – people tend to increase the value of chosen options and to decrease the value of unchosen options. Our work builds upon these studies and extends them to show how the value of an unchosen option changes as a function of the chosen option’s outcome. 

Many questions still remain. Is the relationship between choice options reciprocal: Does learning about the option we passed over change the value of the option we chose? What are the neural mechanisms involved in creating the association between choice alternatives and using it for value transfer? Does value transfer occur during counterfactual thinking (such as thinking, “Hmph I should have chosen that salad!”)? What happens when the decision is not binary? We hope to explore these questions and others in the future.

Illustration by Matteo Farinella, Scientific Multimedia Producer at the Zuckerman Mind Brain Behavior Institute, Columbia University

 

References

  1. Sutton, R. S. & Barto, A. G. Introduction to reinforcement learning. 135, (MIT press Cambridge, 1998).
  2. Biderman, N. & Shohamy, D. Memory and Decision Making Interact to Shape the Value of Unchosen Options. GitHub (2021). doi:10.5281/zenodo.4926568
  3. Wimmer, G. E. & Shohamy, D. Preference by association: How memory mechanisms in the hippocampus bias decisions. Science. 338, 270–273 (2012).
  4. Palminteri, S., Lefebvre, G., Kilford, E. J. & Blakemore, S. J. Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing. PLoS Comput. Biol. 13, e1005684 (2017).
  5. Mather, M., Shafir, E. & Johnson, M. K. Misremembrance of options past: Source Monitoring and Choice. Psychol. Sci. 11, 132–138 (2000).
  6. Henkel, L. A. & Mather, M. Memory attributions for choices: How beliefs shape our memories. J. Mem. Lang. 57, 163–176 (2007).
  7. Sharot, T., Fleming, S. M., Yu, X., Koster, R. & Dolan, R. J. Is Choice-Induced Preference Change Long Lasting? Psychol. Sci. 23, 1123–1129 (2012).
  8. Festinger, L. Conflict, decision, and dissonance. (Stanford University Press, Stanford CA, 1964).
  9. Bem, D. J. Self-perception: An alternative interpretation of cognitive dissonance phenomena. Psychol. Rev. 74, 183 (1967).

 

 

Natalie Biderman

PhD student, Columbia University