Evolutionary game theory
Evolutionary game theory is the application of game theory to evolving populations in biology. It defines a framework of contests, strategies, and analytics into which Darwinian competition can be modelled. It originated in 1973 with John Maynard Smith and George R. Price's formalisation of contests, analysed as strategies, and the mathematical criteria that can be used to predict the results of competing strategies.
Evolutionary game theory differs from classical game theory in focusing more on the dynamics of strategy change. This is influenced by the frequency of the competing strategies in the population.
Evolutionary game theory has helped to explain the basis of altruistic behaviours in Darwinian evolution. It has in turn become of interest to economists, sociologists, anthropologists, and philosophers.
History
Classical game theory
Classical non-cooperative game theory was conceived by John von Neumann to determine optimal strategies in competitions between adversaries. A contest involves players, all of whom have a choice of moves. Games can be a single round or repetitive. The approach a player takes in making his moves constitutes his strategy. Rules govern the outcome for the moves taken by the players, and outcomes produce payoffs for the players; rules and resulting payoffs can be expressed as decision trees or in a payoff matrix. Classical theory requires the players to make rational choices. Each player must consider the strategic analysis that his opponents are making to make his own choice of moves.The problem of ritualized behaviour
Evolutionary game theory started with the problem of how to explain ritualized animal behaviour in a conflict situation; "why are animals so 'gentlemanly or ladylike' in contests for resources?" The leading ethologists Niko Tinbergen and Konrad Lorenz proposed that such behaviour exists for the benefit of the species. John Maynard Smith considered that incompatible with Darwinian thought, where selection occurs at an individual level, so self-interest is rewarded while seeking the common good is not. Maynard Smith, a mathematical biologist, turned to game theory as suggested by George Price, though Richard Lewontin's attempts to use the theory had failed.Adapting game theory to evolutionary games
Maynard Smith realised that an evolutionary version of game theory does not require players to act rationally —– only that they have a strategy. The results of a game shows how good that strategy was, just as evolution tests alternative strategies for the ability to survive and reproduce. In biology, strategies are genetically inherited traits that control an individual's action, analogous with computer programs. The success of a strategy is determined by how good the strategy is in the presence of competing strategies, and of the frequency with which those strategies are used. Maynard Smith described his work in his book Evolution and the Theory of Games.Participants aim to produce as many replicas of themselves as they can, and the payoff is in units of fitness. It is always a multi-player game with many competitors. Rules include replicator dynamics, in other words how the fitter players will spawn more replicas of themselves into the population and how the less fit will be culled, in a replicator equation. The replicator dynamics models heredity but not mutation, and assumes asexual reproduction for the sake of simplicity. Games are run repetitively with no terminating conditions. Results include the dynamics of changes in the population, the success of strategies, and any equilibrium states reached. Unlike in classical game theory, players do not choose their strategy and cannot change it: they are born with a strategy and their offspring inherit that same strategy.
Evolutionary games
Models
Evolutionary game theory encompasses Darwinian evolution, including competition, natural selection, and heredity. Evolutionary game theory has contributed to the understanding of group selection, sexual selection, altruism, parental care, co-evolution, and ecological dynamics. Many counter-intuitive situations in these areas have been put on a firm mathematical footing by the use of these models.The common way to study the evolutionary dynamics in games is through replicator equations. These show the growth rate of the proportion of organisms using a certain strategy and that rate is equal to the difference between the average payoff of that strategy and the average payoff of the population as a whole. Continuous replicator equations assume infinite populations, continuous time, complete mixing and that strategies breed true. The attractors of the equations are equivalent with evolutionarily stable states. A strategy which can survive all "mutant" strategies is considered evolutionarily stable. In the context of animal behavior, this usually means such strategies are programmed and heavily influenced by genetics, thus making any player or organism's strategy determined by these biological factors.
Evolutionary games are mathematical objects with different rules, payoffs, and mathematical behaviours. Each "game" represents different problems that organisms have to deal with, and the strategies they might adopt to survive and reproduce. Evolutionary games are often given colourful names and cover stories which describe the general situation of a particular game. Representative games include hawk-dove, war of attrition, stag hunt, producer-scrounger, tragedy of the commons, and prisoner's dilemma. Strategies for these games include Hawk, Dove, Bourgeois, Prober, Defector, Assessor, and Retaliator. The various strategies compete under the particular game's rules, and the mathematics are used to determine the results and behaviours.
Hawk Dove
The first game that Maynard Smith analysed is the classic Hawk Dove game. It was conceived to analyse Lorenz and Tinbergen's problem, a contest over a shareable resource. The contestants can be either Hawk or Dove. These are two subtypes or morphs of one species with different strategies. The Hawk first displays aggression, then escalates into a fight until it either wins or is injured. The Dove first displays aggression, but if faced with major escalation runs for safety. If not faced with such escalation, the Dove attempts to share the resource.| meets Hawk | meets Dove | |
| if Hawk | V/2 − C/2 | V |
| if Dove | 0 | V/2 |
Given that the resource is given the value V, the damage from losing a fight is given cost C:
- If a Hawk meets a Dove he gets the full resource V to himself
- If a Hawk meets a Hawk – half the time he wins, half the time he loses...so his average outcome is then V/2 minus C/2
- If a Dove meets a Hawk he will back off and get nothing – 0
- If a Dove meets a Dove both share the resource and get V/2
The solution of the Hawk Dove Game explains why most animal contests involve only ritual fighting behaviours in contests rather than outright battles. The result does not at all depend on good of the species behaviours as suggested by Lorenz, but solely on the implication of actions of so-called selfish genes.
War of attrition
In the Hawk Dove game the resource is shareable, which gives payoffs to both Doves meeting in a pairwise contest. Where the resource is not shareable, but an alternative resource might be available by backing off and trying elsewhere, pure Hawk or Dove strategies are less effective. If an unshareable resource is combined with a high cost of losing a contest both Hawk and Dove payoffs are further diminished. A safer strategy of lower cost display, bluffing and waiting to win, is then viable – a Bluffer strategy. The game then becomes one of accumulating costs, either the costs of displaying or the costs of prolonged unresolved engagement. It is effectively an auction; the winner is the contestant who will swallow the greater cost while the loser gets the same cost as the winner but no resource. The resulting evolutionary game theory mathematics leads to an optimal strategy of timed bluffing.This is because in the war of attrition any strategy that is unwavering and predictable is unstable, because it will ultimately be displaced by a mutant strategy which relies on the fact that it can best the existing predictable strategy by investing an extra small delta of waiting resource to ensure that it wins. Therefore, only a random unpredictable strategy can maintain itself in a population of Bluffers. The contestants in effect choose an acceptable cost to be incurred related to the value of the resource being sought, effectively making a random bid as part of a mixed strategy. This implements a distribution of bids for a resource of specific value V, where the bid for any specific contest is chosen at random from that distribution. The distribution can be computed using the Bishop-Cannings theorem, which holds true for any mixed strategy ESS. The distribution function in these contests was determined by Parker and Thompson to be:
The result is that the cumulative population of quitters for any particular cost m in this "mixed strategy" solution is:
as shown in the adjacent graph. The intuitive sense that greater values of resource sought leads to greater waiting times is borne out. This is observed in nature, as in male dung flies contesting for mating sites, where the timing of disengagement in contests is as predicted by evolutionary theory mathematics.
Asymmetries that allow new strategies
In the War of Attrition there must be nothing that signals the size of a bid to an opponent, otherwise the opponent can use the cue in an effective counter-strategy. There is however a mutant strategy which can better a Bluffer in the War of Attrition Game if a suitable asymmetry exists, the Bourgeois strategy. Bourgeois uses an asymmetry of some sort to break the deadlock. In nature one such asymmetry is possession of a resource. The strategy is to play a Hawk if in possession of the resource, but to display then retreat if not in possession. This requires greater cognitive capability than Hawk, but Bourgeois is common in many animal contests, such as in contests among mantis shrimps and among speckled wood butterflies.Social behaviour
Games like Hawk Dove and War of Attrition represent pure competition between individuals and have no attendant social elements. Where social influences apply, competitors have four possible alternatives for strategic interaction. This is shown on the adjacent figure, where a plus sign represents a benefit and a minus sign represents a cost.- In a Cooperative or Mutualistic relationship both "donor" and "recipient" are almost indistinguishable as both gain a benefit in the game by co-operating, i.e. the pair are in a game-wise situation where both can gain by executing a certain strategy, or alternatively both must act in concert because of some encompassing constraints that effectively puts them "in the same boat".
- In an Altruistic relationship the donor, at a cost to himself provides a benefit to the recipient. In the general case the recipient will have a kin relationship to the donor and the donation is one-way. Behaviours where benefits are donated alternatively at a cost, are often called altruistic, but on analysis such "altruism" can be seen to arise from optimised "selfish" strategies
- Spite is essentially a "reversed" form of altruism where an ally is aided by damaging the ally's competitor. The general case is that the ally is kin related and the benefit is an easier competitive environment for the ally. Note: George Price, one of the early mathematical modellers of both altruism and spite, found this equivalence particularly disturbing at an emotional level.
- Selfishness is the base criteria of all strategic choice from a game theory perspective – strategies not aimed at self-survival and self-replication are not long for any game. Critically however, this situation is impacted by the fact that competition is taking place on multiple levels – i.e. at a genetic, an individual and a group level.
Contests of selfish genes
| The Mathematics of Kin Selection | |||||||||
| The concept of Kin Selection is that: Fitness is measured relative to the average population; for example, fitness=1 means growth at the average rate for the population, fitness < 1 means having a decreasing share in the population, fitness > 1 means an increasing share in the population. The inclusive fitness of an individual wi is the sum of its specific fitness of itself ai plus the specific fitness of each and every relative weighted by the degree of relatedness which equates to the summation of all rj*bj....... where rj is relatedness of a specific relative and bj is that specific relative's fitness – producing: Now if individual ai sacrifices his "own average equivalent fitness of 1" by accepting a fitness cost C, and then to "get that loss back", wi must still be 1...and if we use R*B to represent the summation we get: Hamilton went beyond kin relatedness to work with Robert Axelrod, analysing games of co-operation under conditions not involving kin where reciprocal altruism comes into play. Eusociality and kin selectioninsect workers forfeit reproductive rights to their queen. It has been suggested that Kin Selection, based on the genetic makeup of these workers, may predispose them to altruistic behaviour. Most eusocial insect societies have haplodiploid sexual determination, which means that workers are unusually closely related.This explanation of insect eusociality has however been challenged by a few highly noted evolutionary game theorists who have published a controversial alternative game theoretic explanation based on a sequential development and group selection effects proposed for these insect species. Prisoner's dilemmaA difficulty of the theory of evolution, recognised by Darwin himself, was the problem of altruism. If the basis for selection is at individual level, altruism makes no sense at all. But universal selection at the group level fails to pass the test of the mathematics of game theory and is certainly not the general case in nature. Yet in many social animals, altruistic behaviour exists. The solution to this paradox can be found in the application of evolutionary game theory to the prisoner's dilemma game – a game which tests the payoffs of cooperating or in defecting from cooperation. It is certainly the most studied game in all of game theory.The analysis of prisoner's dilemma is as a repetitive game. This affords competitors the possibility of retaliating for defection in previous rounds of the game. Many strategies have been tested; the best competitive strategies are general cooperation with a reserved retaliatory response if necessary. The most famous and one of the most successful of these is tit-for-tat with a simple algorithm. procedure tit-for-tat EventBit:=Trust; do while Contest=ON; if Eventbit=Trust then Cooperate else Defect; if Opponent_Move=Cooperate then EventBit:=Trust else Eventbit:=NOT; end; The pay-off for any single round of the game is defined by the pay-off matrix for a single round game. In multi-round games the different choices – Co-operate or Defect – can be made in any particular round, resulting in a certain round payoff. It is, however, the possible accumulated pay-offs over the multiple rounds that count in shaping the overall pay-offs for differing multi-round strategies such as Tit-for-Tat. game. Prisoner's dilemma: Co-operate or Defect? Payoff > Payoff > Payoff > Payoff Example 1: The straightforward single round prisoner's dilemma game. The classic prisoner's dilemma game payoffs gives a player a maximum payoff if he defect and his partner co-operates. If however the player co-operates and his partner defects, he gets the worst possible result. In these payoff conditions the best choice is to defect. Example 2: Prisoner's dilemma played repeatedly. The strategy employed is Tit-for-Tat which alters behaviors based on the action taken by a partner in the previous round – i.e. reward co-operation and punish defection. The effect of this strategy in accumulated payoff over many rounds is to produce a higher payoff for both players co-operation and a lower payoff for defection. This removes the Temptation to defect. The suckers payoff also becomes less, although "invasion" by a pure defection strategy is not entirely eliminated. Routes to altruismAltruism takes place when one individual, at a cost C to itself, exercises a strategy that provides a benefit B to another individual. The cost may consist of a loss of capability or resource which helps in the battle for survival and reproduction, or an added risk to its own survival. Altruism strategies can arise through:
The evolutionarily stable strategyThe evolutionarily stable strategy is akin to Nash equilibrium in classical game theory, but with mathematically extended criteria. Nash Equilibrium is a game equilibrium where it is not rational for any player to deviate from his present strategy, provided that the others adhere to their strategies. An ESS is a state of game dynamics where, in a very large population of competitors, another mutant strategy cannot successfully enter the population to disturb the existing dynamic. Therefore, a successful strategy must be both effective against competitors when it is rare – to enter the previous competing population, and successful when later in high proportion in the population – to defend itself. This in turn means that the strategy must be successful when it contends with others exactly like itself.An ESS is not:
Let the chance of meeting a Hawk=p so therefore the chance of meeting a dove is Let WHawk equal the Payoff for Hawk..... WHawk=Payoff in the chance of meeting a Dove + Payoff in the chance of meeting a Hawk Taking the PAYOFF MATRIX results and plugging them into the above equation: Similarly for a Dove: so.... Equating the two fitnesses, Hawk and Dove ... and solving for p so for this "static point" where the Population Percent is an ESS solves to be ESS=V/C Similarly, using inequalities, it can be shown that an additional Hawk or Dove mutant entering this ESS state eventually results in less fitness for their kind – both a true Nash and an ESS equilibrium. This example shows that when the risks of contest injury or death is significantly greater than the potential reward, the stable population will be mixed between aggressors and doves, and the proportion of doves will exceed that of the aggressors. This explains behaviours observed in nature. Unstable games, cyclic patternsRock paper scissorsAn evolutionary game that turns out to be a children's game is rock paper scissors. The game is simple – rock beats scissors, scissors beats paper, and paper beats rock. Anyone who has ever played this simple game knows that it is not sensible to have any favoured play – the opponent will soon notice this and switch to the winning counter-play. The best strategy is to play a mixed random game with any of the three plays taken a third of the time. This, in evolutionary game theory terms, is a mixed strategy. But many lifeforms are incapable of mixed behavior – they only exhibit one strategy. If the game is played only with the pure Rock, Paper and Scissors strategies the evolutionary game is dynamically unstable:Rock mutants can enter an all scissor population, but then – Paper mutants can take over an all Rock population, but then – Scissor mutants can take over an all Paper population – and on and on.... This is easily seen on the game payoff matrix, where if the paths of mutant invasion are noted, it can be seen that the mutant "invasion paths" form into a loop. This in triggers a cyclic invasion pattern. Rock paper scissors incorporated into an evolutionary game has been used for modelling natural processes in the study of ecology. Using experimental economics methods, scientists have used RPS game to test human social evolutionary dynamical behaviors in laboratory. The social cyclic behaviors, predicted by evolutionary game theory, have been observed in various laboratory experiments. The side-blotched lizardThe side-blotched lizard is polymorphic with three morphs that each pursues a different mating strategyeffectively uses a rock-paper-scissors mating strategy. However the blue throats cannot overcome the more aggressive orange throats. The overall situation corresponds to the Rock, Scissors, Paper game, creating a six-year population cycle. When he read that these lizards were essentially engaged in a game with rock-paper-scissors structure, John Maynard Smith is said to have exclaimed "They have read my book!" Signalling, sexual selection and the handicap principleAside from the difficulty of explaining how altruism exists in many evolved organisms, Darwin was also bothered by a second conundrum – why do a significant number of species have phenotypical attributes that are patently disadvantageous to them with respect to their survival – and should by the process of natural section be selected against – e.g. the massive inconvenient feather structure found in a peacock's tail? Regarding this issue Darwin wrote to a colleague "The sight of a feather in a peacock's tail, whenever I gaze at it, makes me sick." It is the mathematics of evolutionary game theory, which has not only explained the existence of altruism but also explains the totally counterintuitive existence of the peacock's tail and other such biological encumbrances.On analysis, problems of biological life are not at all unlike the problems that define economics – eating, survival and reproduction. Game theory was originally conceived as a mathematical analysis of economic processes and indeed this is why it has proven so useful in explaining so many biological behaviours. One important further refinement of the evolutionary game theory model that has economic overtones rests on the analysis of costs. A simple model of cost assumes that all competitors suffer the same penalty imposed by the Game costs, but this is not the case. More successful players will be endowed with or will have accumulated a higher "wealth reserve" or "affordability" than less successful players. This wealth effect in evolutionary game theory is represented mathematically by "resource holding potential " and shows that the effective cost to a competitor with higher RHP are not as great as for a competitor with a lower RHP. As a higher RHP individual is more desirable mate in producing potentially successful offspring, it is only logical that with sexual selection RHP should have evolved to be signalled in some way by the competing rivals, and for this to work this signalling must be done honestly. Amotz Zahavi has developed this thinking in what is known as the handicap principle, where superior competitors signal their superiority by a costly display. As higher RHP individuals can properly afford such a costly display this signalling is inherently honest, and can be taken as such by the signal receiver. Nowhere in nature is this better illustrated than in the magnificent and costly plumage of the peacock. The mathematical proof of the handicap principle was developed by Alan Grafen using evolutionary game-theoretic modelling. CoevolutionTwo types of dynamics have been discussed so far in this article:
In competitive inter-species coevolutionary system the species are involved in an arms race – where adaptations that are better at competing against the other species tend to be preserved. Both game payoffs and replicator dynamics reflect this. This leads to a Red Queen dynamic where the protagonists must "run as fast as they can to just stay in one place". A number of evolutionary game theory models have been produced to encompass coevolutionary situations. A key factor applicable in these coevolutionary systems is the continuous adaptation of strategy in such arms races. Coevolutionary modelling therefore often includes genetic algorithms to reflect mutational effects, while computers simulate the dynamics of the overall coevolutionary game. The resulting dynamics are studied as various parameters are modified. Because several variables are simultaneously at play, solutions become the province of multi-variable optimisation. The mathematical criteria of determining stable points are Pareto efficiency and Pareto dominance, a measure of solution optimality peaks in multivariable systems. Carl Bergstrom and Michael Lachmann apply evolutionary game theory to the division of benefits in mutualistic interactions between organisms. Darwinian assumptions about fitness are modeled using replicator dynamics to show that the organism evolving at a slower rate in a mutualistic relationship gains a disproportionately high share of the benefits or payoffs. Extending the modelA mathematical model analysing the behaviour of a system needs initially to be as simple as possible to aid in developing a base understanding the fundamentals, or “first order effects”, pertaining to what is being studied. With this understanding in place it is then appropriate to see if other, more subtle, parameters further impact the primary behaviours or shape additional behaviours in the system. Following Maynard Smith's seminal work in evolutionary game theory, the subject has had a number of very significant extensions which have shed more light on understanding evolutionary dynamics, particularly in the area of altruistic behaviors. Some of these key extensions to evolutionary game theory are:Spatial GamesGeographic factors in evolution include gene flow and horizontal gene transfer. Spatial game models represent geometry by putting contestants in a lattice of cells: contests take place only with immediate neighbours. Winning strategies take over these immediate neighbourhoods and then interact with adjacent neighbourhoods. This model is useful in showing how pockets of co-operators can invade and introduce altruism in the Prisoners Dilemma game, where Tit for Tat is a Nash Equilibrium but NOT also an ESS. Spatial structure is sometimes abstracted into a general network of interactions. This is the foundation of evolutionary graph theory.Effects of having informationIn evolutionary game theory as in conventional Game Theory the effect of Signalling is of critical importance, as in Indirect Reciprocity in Prisoners Dilemma. This models the reality of most normal social interactions which are non-kin related. Unless a probability measure of reputation is available in Prisoners Dilemma only direct reciprocity can be achieved. With this information indirect reciprocity is also supported.Alternatively, agents might have access to an arbitrary signal initially uncorrelated to strategy but becomes correlated due to evolutionary dynamics. This is the green-beard effect or evolution of ethnocentrism in humans. Depending on the game, it can allow the evolution of either cooperation or irrational hostility. From molecular to multicellular level, a signaling game model with information asymmetry between sender and receiver might be appropriate, such as in mate attraction or evolution of translation machinery from RNA strings. Finite populationsMany evolutionary games have been modelled in finite populations to see the effect this may have, for example in the success of mixed strategies. |