When deciding between more than two options, people often show perplexing choice anomalies. One of such paradoxes was introduced in a recent study by Louie and colleagues (2013), in which participants made decisions between three food snack options. The authors reported that the ability to choose the better out of the two most attractive snacks depended on how bad the third and worst option was. If the third snack was much worse than the other two snacks, then it was easy to pick the best option. If the third snack was almost as good as the other alternatives, then participants had a much harder time to choose their favorite snack.
When reading the paper by Louie and colleagues, my students and I became fascinated by this “distractor effect” (distractor, because the third option acts as a distractor in the decision between the two better options). However, we had a completely different explanation for it than the one put forward by the authors. While the authors attributed the distractor effect to some fundamental principles of how neurons in the brain are supposed to encode the subjective values of different choice options, we favored a cognitive (and arguably more mundane) explanation of the distractor effect.
More specifically, we had attention in mind. When making decisions, people need to distribute their attention across all options. And because attention is a limited capacity, people should try to ignore irrelevant options, such as bad third options. But the ability to identify and then ignore bad third options should depend on how bad this option is. The worse it is, the easier it is to ignore. This would reserve more attentional capacity for choosing among the two better options, which could make it easier to pick the best option and explains the distractor effect.
Thus, the path for our study was set. We would use eye tracking to record participants’ eye movements while they made decisions between three food snacks. Then we would test the hypothesis that participants try to ignore third options and succeed with this only if the third option was much worse than the other two options. This in turn would be related to the probability to pick the best option.
But there was one more thing to take care of, and this was the question of whether the distractor effect is replicable. Notably, the effect reported by Louie and colleagues was a novel choice paradox and has never been subjected to a direct replication attempt so far. Therefore, we decided to perform such a direct replication of the distractor effect in addition to the eye-tracking experiment. To match the procedures of the original experiment as much as possible, we contacted the first author of the study, Kenway Louie. We were very fortunate that Kenway Louie collaborated with us in an exemplary and extremely cooperative way. He sent us a lot of study materials and replied to all our questions and requests.
When analyzing the data, we were surprised by the results. One the one hand, our predictions about the allocation of attention were correct. Participants tried to ignore the worst option, and their success in doing so depended on how bad the worst option was. On the other hand, we could not replicate the distractor effect. That is, bad third options did not increase the chance of choosing the best out of the two good options. All our analyses (which included replication analyses based on frequentist statistics, Bayesian analyses and model comparisons) pointed towards strong evidence for the null hypothesis and an unquestionably failed replication.
However, we found that bad third options allowed participants to make faster decisions, and we were able to account for this finding (together with the null finding on choice probability and the eye-tracking results) by extending an established cognitive model of attention and decision making (Krajbich and Rangel, 2011). So, in the end the results made a lot of sense to us – but it would be wrong to claim that we expected all of this at the outset. This is just how science really works.
Krajbich, I. & Rangel, A. Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc. Natl. Acad. Sci. 108, 13852– 13857 (2011).