The Evolution of a Unified Framework of Reciprocity

When deciding whether to cooperate with someone, would people usually rely on their own experience with that person, or would they rather refer to that person’s public reputation? Our model suggests that the answer depends on the environment in which these decisions take place.
The Evolution of a Unified Framework of Reciprocity

Most humans are conditional cooperators. When asked for help, they are more likely to grant a favor if the other person is known to be generous too. The scientific field that models the emergence of such conditional behaviors is evolutionary game theory. It considers populations of interacting individuals, called players, who make decisions based on their strategies. The strategies tell the individuals how to act in a given situation. In the context of cooperation, individuals might have the strategy to always help; they might choose to never help; or they may act conditionally, depending on the interaction partner’s previous behavior. When individuals preferentially help those who have been helpful in the past, they engage in reciprocity. 

The evolutionary game theory literature distinguishes two kinds of reciprocity. According to direct reciprocity, we should help those who helped us in the past (or those who are likely to return the favor in the future). According to indirect reciprocity, we should help those with a good reputation, for example, because they helped someone else in the past. Interesting conflicts arise if the two kinds of reciprocity make competing suggestions. Should you help someone who has treated you unfairly, but who is known to be a good person in general? 

Figure 1: A comparison of direct and indirect reciprocity.

Although this type of question perfectly fits into the scope of evolutionary game theory, it is difficult to answer. One reason for this difficulty is that the existing models of indirect reciprocity are strikingly different from models of direct reciprocity. In most models of direct reciprocity, the strategies are remarkably simple. For example, according to the perhaps most well-known strategy, Tit-for-Tat, individuals should simply consider their last interaction with the respective co-player. If the co-player cooperated, one should cooperate. Otherwise, one should defect. In contrast, the strategies in indirect reciprocity take the form of elaborate social norms. These norms describe how to evaluate the reputation of a given co-player at any point in time. In most models, these evaluations depend on the co-player’s previous actions, but also on the reputation of the co-player’s previous co-players (which in turn depends on those players’ last co-players). As one can easily imagine, describing the dynamics of indirect reciprocity quickly becomes a rather complex endeavor.  

When Krish, Martin, and I started thinking about these questions, we had just finished a related project. For that project, we explored the evolution of direct reciprocity with "crosstalk". In that model, individuals would usually react to the past behavior of their immediate co-player. However, occasionally they would also be affected by their experiences in other, unrelated games. We realized that a similar model may also be capable of describing the interaction of direct and indirect reciprocity. Around this time, Laura started a PhD rotation in Krish’s group. We all thought that a unified framework of reciprocity, if possible, would make for a great PhD thesis. 

A few weeks into the rotation, Laura had already derived the main equation that we used to analyze the interplay between direct and indirect reciprocity. Our idea was to start from a typical model of reciprocity. For this model, we considered so-called reactive strategies (i.e., all strategies that have the same complexity as Tit-for-Tat). In addition, we added another variable that would measure to which extent individuals consider outside information. If this variable takes a value of zero, we have a model of direct reciprocity. If it takes a value of one, we recover the most elementary model of indirect reciprocity. 

This framework is useful for two reasons. First, it allowed us to use mathematical tools developed for direct reciprocity to explore questions of indirect reciprocity. In particular, Laura was quickly able to show that a prominent strategy of direct reciprocity, Generous Tit-for-Tat, has a natural analogue in indirect reciprocity, which we called Generous Scoring. We also proved that Generous Scoring is a Nash equilibrium. That is, once the whole population adopts this norm, no population member would like to switch to a different norm, not even if the alternative norm is arbitrarily more sophisticated. 

Second, the framework also allowed us to explore which kind of reciprocity individuals would prefer, if given the choice. Using individual-based simulations, we show that direct reciprocity is favored when individuals have many interactions with each other, or when outside information is particularly noisy. In contrast, indirect reciprocity can sustain cooperation even when individuals only meet occasionally. However, indirect reciprocity requires the evolutionary process to be faithful, such that mutations are sufficiently rare. 

Although we have explored many additional model extensions, our model is only a first step towards a complete description of direct and indirect reciprocity. Interesting challenges lie ahead. For example, our model suggests that the benefits of indirect reciprocity are nonlinear in the number of cooperators. That is, for a social norm to have a noticeable effect, it needs to be adopted by sufficiently many. Whether this conclusion holds for all social norms, and which norm is most effective, are a few of the questions that we hope will be tackled in the future.