Diplomacy and Game Theory

Part 2: Rational Choice

David C. Rosen

This paper will explore the application of Game Theoretical concepts to the game Diplomacy. After an brief introduction of the game ¹, the paper will discuss to what extent Diplomacy conforms to the methodological requirements of rational choice analysis and Game Theoretical analysis, and discuss where problems arise. The paper will then discuss what kind of game Diplomacy is best described as, suggesting that its overall n-person zero-sum game characteristics can be subdivided at points into other sub-games of different types. The paper will finally present a few examples of how Game Theory can be applied to Diplomacy.

Note from the author: This paper does not provide the promised sequel to my first article. That is it will not cover non-game theoretical approaches in IR (such as Robert Jervis' Theory of Misperception). It will instead be just another game theoretical paper. But it is in my opinion far better than the first. That early assay into unknown waters displayed some interesting failures to grasp basic concepts. Informed by a year of hard-core Rational Choice courses, this one should promise more.

Introduction

The game Diplomacy, in its standard form as sold by Hasbro, is a seven-player strategy game played on a map of pre-First World War Europe ². In Diplomacy, there are no dice or other instruments of chance; the outcome of every move is determined by a combination of each players actions, or, in the games' terms, their orders. The object of the game is to expand to control more than half the power centers on the map. For practical purposes, winning is only possible through cooperation with other players ³ - each of whom seeks the same goal. While draws are possible, and unavoidable in stalemate positions, winning is the goal ⁴. There is thus a basic tension in Diplomacy: cooperation is all but essential to victory, but victory is only possible at the expense of those whose cooperation the would-be winner seeks.

The objective of the game, and its condition of systemic anarchy (there is no higher power than the players beyond the technical rules of the game) provide a Hobbseian condition undoubtedly purer than the international reality deduced by the harshest of neorealist theorists ⁵. Moreover, the lack of the status quo as a preferred final outcome for any player (as opposed to in the real world) ⁶ coupled with a normative environment that typically admires guile, deviousness and ruthlessness, drastically limits the power of moral constraints in mitigating the state of nature. Keeping in mind that as in all zero-sum games, others' gains and losses effect the relative position of each player, the simplifying assumption of egoism requires little justification in Diplomacy.

Diplomacy and the Methodological Postulates of Rational Choice Theory

Before plunging into a detailed discussion of how Game Theory can be used to understand strategies and tactics used within Diplomacy, it is first desirable to ascertain to what extent the game conforms the necessary methodological assumptions of Game Theory and its parent, Rational Choice Theory. To the extent that does not, all such analyses will risk a priori invalidation.

The two fundamental underpinning assumptions of Rational Choice are methodological individualism and purposeful action ⁷. Methodological individualism simply means that outcomes can be understood by reference to individual preferences and choices, rather than through institutional characteristics or collective units of analysis. To analyze real political events with Rational Choice Theory, making this assumption necessitates coming down firmly on one side of a great methodological debate; to so analyze Diplomacy, no such commitment is necessary. Players are individuals, and though decisions as to which orders to give may be influenced by other players, the institution that provides the basic level of input which leads to outcomes on the "macro" scale of the game is the player. The analyst will find no barrier here to further examination of Diplomacy.

Purposive action is simply the assumption of rationality itself. Actions are aimed at achieving outcomes, and all outcomes have transitive preference relationships to each other. A rational player will always choose those actions that he believes will lead to the outcome that he most prefers, of those possible. Diplomacy players on the whole (there are always exceptions of course) would seem to offer little difficulty to the rationality assumption, assuming reasonably that they have read the rules well enough to understand which actions lead to which outcomes.

Game Theory and the Game

The main characteristic of a case suitable for Game Theoretical analysis is that outcomes are determined by the actions of a player and the actions of other players (and possibly of nature). Particular payoffs/outcomes correspond to each combination of the players' strategies. The further assumption is made, a corollary to the purposive action of Rational Choice, that players try to maximize their payoffs, and that they assume that other players do the same. Nothing here violates either the design or practice of Diplomacy play. It is this basic interaction between the actions taken by each player and the outcome experienced by all players that make Diplomacy a fit topic for Game Theoretical analysis.

However, there may be a problem with transitivity when applied to the expectations of a player in assuming the motivations of other players. Given that each player knows the other players' preference rankings and has a preference ranking which is known to all other players (and that each player knows that the others know that player's preference rankings, and so on) - possible outcomes and payoffs are common knowledge - a Game Theoretical analysis is possible. While all rational players should have transitive preferences, problems arise if one or more players have non-standard preference rankings (i.e., other those stipulated by the rules). Though a player with non-standard preference rankings will still likely conform to the rationality assumption necessary for Rational Choice analysis (the non-standard preferences may still be transitive, and the players actions will still aim to achieve the best possible outcomes as he sees them), the effect will hinder the attempts of other players and of the analyst to analyze the behavior of the player. The standard preferences are as follows:

Winning
2 - way draw
3 - way draw
4 - way draw
5 - way draw
6 - way draw
Survival

Elimination

However, although the official rules state that the object of the game is to win, individual players may prefer a long-lasting alliance that culminates in a draw, ruling out actions that would lead to even certain victory. Since there is more than small incentive for a player to give the impression that one is such a player, most other players assume that all player's have the standard preferences. The nature of the game is such that the presence of a draw-preferring player (a "carebear" in the hobby's jargon) will violate the rationality assumption at least insofar as other players would analyze his behavior. For his actions, rather than being aimed at achieving his "most preferred" outcome (as others will see) will in fact be aimed at achieving a "lesser" outcome.

Even if this player declares his true preferences, the others will assume, as well they might, that the true carebear is just another "cutthroat" like themselves. Thus although a player with non-standard preference rankings will not be irrational from his own perspective, the nature of the game will confound the attempts of other players - and the analyst - to predict his behavior. The practical effect is the same as if the player was actually irrational as that player's actions will not be aimed at achieved at aiming what others' expect to be his most preferred outcome - winning. Of course, it might be possible for players to incorporate the probability that other players have non-standard preferences into their considerations. But this, while not making analysis impossible, makes it extremely complex. For the purposes of this paper, the models given assume that all players have standard preference rankings ⁸.

Another requirement of standard Game Theory is that preferences remain fixed throughout the game. While the actions of others interacting with the actions of the player must effect the outcome for Game Theory to be applicable ⁹, actions of other players should not effect the preferences of the player. Game Theoretical analysis will be distorted if a particular player's preferences change in the course of game due to the actions of others. This typically happens when a player holds a grudge, or suffers from "sour grapes" syndrome. For example, if player A attacks player B, and player B's preference rankings change so that B prefers elimination to surviving if this survival involves Player A's victory (or even survival), the analyst is stymied.

Again, there is some incentive for every player to give the impression that his preferences will change in just this way, to make threats credible in the common cases where carrying out threat will lead to the threatener's elimination. Thus the presence of a grudge-holding player - given that true preferences at variance with the standard preferences are unlikely to be believed - frustrates rational analysis in the same way that does the presence of a player with constant but non-standard preferences. And again, the models here assume players with constant preference rankings.

What Kind of Game is Diplomacy?

First and foremost, if players accept the official objective of winning, Diplomacy is a zero-sum game. Only one player can win, and if he does, all others lose. The specific assets by which victory - and capabilities - is determined, power centers, are of a fixed and limited number (34 on the standard map). Except in the beginning, when some power centers are neutral, the gains of one player directly translate in the losses of at least one other. Even in the beginning, struggles over neutral centers, at least when looked at from the perspective of all players, are of a zero-sum character (e.g., if France gets Belgium, this means that Germany or England does not).

However, though Diplomacy is in fact a zero-sum game, it is not a 2-person zero-sum game (unless all other players but two have been eliminated). This significantly limits the predictive power of the analyst, and of the players. All 2-person zero-sum games have equilibria in pure or mixed strategies. Moreover, all equilibria are equivalent and interchangeable ¹⁰. This does not hold for n-person zero-sum games ¹¹. First, many equilibria may exist for which the results of each are not equivalent to the other. Second, the particular strategies that lead to a given equilibria will not be the same as those that lead to another. Since players have no way of determining which equilibria will be reached, they have no way of knowing which strategies will lead to an equilibria, and will in most cases have no way of determining with certainty which strategy would maximize their utility given the strategies of the other players. Thus, while describing Diplomacy as an n-person zero-sum game is accurate, this offers little more than the possibility of making broad generalizations of little use to players or analysts wishing to make predictions.

Rather than end the discussion on this note of futility, it may be possible to move forward with Game Theoretical analysis by making an assumption. This is that the whole of game Diplomacy can be usefully broken down into a set of sub-games that in themselves differ from the characteristics of n-person zero-sum games. In the interactions between subgroups of players, strategizing may be functionalized by reference to other sorts of games - 2-person zero-sum, 2- and n-person games of conflict and cooperation, even 2-person and n-person games of coordination. All manner of games be taking place simultaneously, bearing in mind that they all fit together under and are constrained by the overall n-person zero sum game.

In Diplomacy, outright conflicts with other players have the aspect of 2-person zero-sum games. The efforts of two wary players to form a new alliance often follow the pattern of iterated cooperative games, where reciprocity - or outside threat - may succeed in building trust to the point where two allies can coordinate their moves in order to make the best use of all their units. For example, Turkey might consider his dealings with an attacking and uncommunicative Russia as a two-person zero-sum game, while attempting to overcome Prisoner's Dilemma logic in establishing an alliance with Austria. Meanwhile England - facing elimination - might feel the need to coordinate perfectly his moves with Germany and Italy against an ascendant France. And yet the overall structure of the n-person zero sum game holds dynamics which often lead to players reconsidering which sub-games they are playing in reference to the overall situation. So Turkey may pause in his successful campaign to gain at Russia's expense if this will lead to Austria's victory, and England, once secure, might change from coordinating moves with Germany to plotting against him, once the threat from France has been beaten back, and Italy may re-frame his understanding of cooperative agreement with Austria after the Kaiser's armies begin pouring over the border into a 2-person zero-sum game of minimizing losses.

What makes a good player, or a good analyst, of Diplomacy is the ability to ascertain which sub-games are appropriate for understanding and thus strategizing in a given game situation. To act as if you are in a game of coordination with a neighbor when in fact he is thinking in zero-sum terms, with your losses being his gains, is to invite disaster. Conversely, failing to see that another player is about to achieve victory might cause one to continue falling into the trap of Prisoner's Dilemma logic with those with whom the best and most urgent strategy would call for coordination.

The appropriate games to be played are a function of expected benefits of strategies, given which games other players are thought to be playing for that game turn. In certain situations, such as when one player is about to win the game, the sub-games are clear, and players tend to coordinate against the pretender, while the would-be victor attempts to maximize his chances of victory utilizing mixed and pure strategies within the context a 2-person zero-sum game of him against a grand coalition. The pretender often holds an advantage aside from any positional details in that he need only coordinate his side of the zero-sum within his head, while those that would prevent his victory must play a game of coordination with each other at the same time as they as a group must face the pretender. On the other hand, a group, given the inherent limitations in analyzing often immensely complicated possibilities, may hold an advantage in that many minds may find solutions that one may miss, especially under time constraints that exacerbate bounded rationality.

Some Game Theory Models of Diplomacy

The ways of using Game Theory models to analyze situations in Diplomacy are all but infinite. One of the simplest could examine mixed strategy calculations designed to overcome frequent situations where no one strategy dominates others in a 2-person zero-sum tactical situation. Mixed strategies are generally superior to pure maximin strategies in the tactics of Diplomacy as they tend provide higher benefits. For example, a rare but occasional opening order set for Italy includes a move from Venice to Piedmont, threatening Marseilles and leaving France with a quandary, especially if he has opened with the very common move from Marseilles to Spain. France risks losing Marseilles, but if he simply moves back using Maximin logic, a laughing Italy can hold his army and watch as France loses the opportunity to gain Spain in the first turn, and loses the possibility of building in Marseilles in order to launch a counter-attack. Limiting the analysis to the two units concerned, and assuming (as is reasonable after a surprise attack by Italy on France in the first turn) that both players view the game as a 2-person zero-sum game offers a simple example of how mixed strategies add to a player's chance of success in Diplomacy. The payoffs given are arbitrary, but in relative terms hopefully do not stray too far from the actual consequences of the various outcomes. The number reflects the payoff to France, the opposite of which is for Italy:

Orders:		Italian Army in Piedmont
Orders:		To Mar	Holds
French Army in Spain	to Mar	0	-2
French Army in Spain	Hold	-5	0

If France holds to a pure strategy maximin strategy and simply orders back to Marseilles, and Italy expects this, Italy can simply hold to guarantee a payoff of 2. Yet France will not profit by simply flipping a coin: if Italy perceives this strategy, he will surely order to Marseilles as the expected payoff is 2.5 [1/2(-5)+1/2(0)], even higher than if France had followed a pure-strategy maximin strategy. Yet the simplicity of 2-person zero-sum games with only two options for each player allows for the best strategy to be determined algebraically (more complicated situations can be solved with calculus). From France's point of view, his losses are minimized if he moves to Marseilles with a probability of P, and probability P is maximized in respect to Italy's possible options.

On the graph, lines derived from the strategy sets intersect over the best strategy for France, which is that he should move to Marseilles 5/7 of the time:

0P -2(1-P)	=	-5P+0(1-P)
P	=	5/7

This strategy yields an expected value to France of -10/7, no matter what strategy Italy follows. However, Italy is, faced with this logic, is best advised to move to Marseilles 2/7 of the time (the game is symmetrical), for if France anticipates that Italy will choose another strategy, France will have incentive to change to a pure strategy. For example, if Italy decides to move to Marseilles 1/2 the time, France can achieve an expected payoff of -1 if he moves to Marseilles with a probability of 1 [1/2(0)+1/2(-2)].

Although the previous model may be of use to players in maximizing their gains in uncertain situations (of which Diplomacy is full), the model is not very interesting for the analyst. It would be much more interesting to derive a model that went some way toward understand the basic flow of Diplomacy from opportunistic alliance to back-stabbing to last-ditch efforts at coordinating against a would-be winner. To do so in anything near a complete manner would take more paper than exists, as the possible permutations involved in just one Diplomacy move stretch into the trillions. Something in between the simple model that explains little more than a tactical technique and a complete one that explores the full strategic dynamics of the game is therefore needed. By limiting the players to three (which occurs if four players are eliminated, as well as with some Diplomacy variants) ¹², and the action sets to strategic decisions, even limiting these to less that than the conceivable at four, a model is presented that, while only scratching the surface of the games' logic, may illuminate.

Consider a game of Diplomacy between players A, B, and C, each controlling powers in symmetrical positions. The following strategies for player A against players B and C to be played simultaneously:

s₁₁=Hold (attack nor support neither opponent)
s₁₂=Support B against C, Attack C
s₁₃=Attack B, Support C against B
s₁₄=Attack both

For player B:

s₂₁=Hold (attack nor support neither opponent)
s₂₂=Support A against C, Attack C
s₂₃=Attack A, Support C against A
s₂₄=Attack both

For player C:

s₃₁=Hold (attack nor support neither opponent)
s₃₂=Support A against B, Attack B
s₃₃=Attack A, Support B against A
s₃₄=Attack both

The expected values of attacking are in an arbitrary but ranked relatively scaled point system as follows:

1 + 2 if supported - 1 if facing a reciprocal attack.

As the game is zero-sum, gains by the attacker are equal to losses by the player attacked. For example, with the strategy set {s₁₃, s₂₂, s₃₄} the result for A is 0 (1 for attack against B -1 from attack by C), for B is -3 (-1 from attack by A, 0 from attack on C, -2 from the supported attack by C), and the payoff for C is 3 (1 from the attack on A, 2 from the supported attack on C).

With even this limited set of strategies, there are 64 possible outcomes. This is barely manageable in extended form (see picture below). An analysis of the game tree provides some insights into the basic logic of a Diplomacy turn. At any point on the tree, the dominant strategy for each player is to attack both of the others. As the game is symmetrical, this results in an equilibrium strategy set of {s₁₄, s₂₄, s₃₄} yielding a payoff to each player of 0. Yet this is not a better outcome than what can be achieved by two players working in concert against the third. For example {s₁₂, s₂₂, s₃₄} yields a payoff of 2,2, and -4 for A, B, and C respectively. However, any attempt to cooperate is hindered by fears that the other player will take advantage of the support to make an even larger gain. If player B plays s₂₄ instead of s₂₂ the payoffs will change to 0, 3, -3, and of course player B rationally prefers 3 to 2. The temptation to defect exists for all would-be coalition partners. It is not possible to overcome this Prisoner's dilemma logic in a one-shot, or rather a one turn game, since normal methods of escaping the problematic, such as moral constraints or intervention of a super powerful actor (e.g., the state), are not available within the context of Diplomacy (and such things would ruin the game or at very least change it beyond recognition).

However, Diplomacy games do not typically begin and end in non-stop stalemate. The fact Diplomacy is played over many turns (typically between 8 and 20 game years, each consisting of 2 movement turns), allow for the Prisoner's Dilemma to be escaped at least temporarily. Most import in this context is perhaps the concept of reciprocity you help me this turn, I help you the next, and so forth.

Another concept that may enable cooperation is the existence of heuristic foci that experienced players may use to reduce the uncertainty of picking which players will make reliable allies. This can simply be reading the way someone speaks, or perhaps more reliably, taking a good look at the map. The designers of standard Diplomacy have evidently placed much effort into making the positions of each power roughly equal. Even so, certain relationships are more natural grounds for alliance than others, especially in the beginning of the game. For example, Italy and Austria make natural allies. Austria faces all but certain elimination at the hands of Turkey and Russia if he attacks Italy at the start. Italy, though he may make quick gains from a quick stab of Austria, will soon face the same problem in protecting his winnings. As the benefits of stabbing each other are in this manner less than going against others, there is an inherent trust possible in an early Austro-Italian alliance.

Finally, this model assumes powers in symmetric positions, something that is fairly rare in Diplomacy play. Actually, when they do occur, the game often ends in a draw, as players calculate the benefits of a 3-way draw compared to the chances of attacking for a win and risking elimination. In many situations, however, unequal power distributions can lead to greater possibilities for cooperation. Particularly when one power is about to achieve victory, the incentive for cooperation and even detailed coordination of moves becomes much greater.

To see this in terms of the model, consider a situation where Player C will win in the next turn if he receives and result better than -3. This could be a reflection of greater power and a small leap needed for victory. In terms of the overall game, players A and B lose more in any outcome set that involves a result for C better than -3, than any payoffs A or B can gain, since they lose the game. In this situation, the strategy set {s₁₄, s₂₄, s₃₄} is no longer an equilibrium. Rather, in this case {s₁₂, s₂₂, s₃₄} becomes the equilibrium with payoffs of 2,2,-4. If either A or B deviates from this strategy set, the result is a victory for C. For example, {s₁₂, s₂₄, s₃₄} leads to -1,3,-2. This is understandable in reference to the overall preference ranking in Diplomacy, where all outcomes save elimination are prefered to survival (the usual result of losing without elimination). However, if for example player A faces elimination if he receives some result lower than 3, {s₁₂, s₂₂, s₃₄} is also no longer an equilibrium. In this case, player A may choose to cooperate with C (which C has an incentive to offer since he need only receive a result higher than -3) to achieve survival over elimination even if this leads to C's victory.

Conclusion

This paper attempts little more than analysis of a board game, and even here only scratches the surface of set of endless puzzles. Yet in the process two points may have been gained. The first is that complicated situations may be modeled in useful ways at the risk of oversimplification but with the possible gain of insight. Diplomacy unlike chess, cannot be solved, but there are resources - for instance experience, guile, and not least of the ability of rational calculation - that enable players to play better and occasionally win. The second point is more a caveat for the first. Rational analysis may be useful, but it always requires assumptions about preferences and human nature that may be misapplied or even flat-out wrong. The analyst, whether examining tic tac toe or the Cuban Missile Crisis, must always tread with caution, and keep in mind that formal symbols and mathematical arguments are built on assumptions - which have no more a priori claim to truth than other assumptions. Assumptions for the purpose of explanation should be made as cautiously as possible, and should be the first points of reflection when analysis fails to predict ¹³,

Footnotes

The rules can be obtained from Hasbro.
The basic rules can be and have been applied to countless other maps. On example covers only British Isle; another includes all of Eurasia. Most variants keep to the basic rules provided with the standard map, and all versions require a minimum of three players, lest the game requires no diplomacy.
Alliance theory, an obvious approach to many aspects of Diplomacy, is covered to a previous paper pp. 4-6: The Game Diplomacy and International Relations Theory.
The official rules state winning as the goal, though particular players may have personal preferences that vary from this norm. This problematic will by discussed in the body of the paper.
The theory of Kenneth Waltz (and to a less systemized extent Realist theory), which asserts that the international environment's basic condition is anarchy, and that all states, as fundamentally like-units, attempt to maximize the security by maximizing their power, seems to this author much more convincing when applied to Diplomacy than when applied to the real thing. See A Theory of International Politics, (Reading: Addision-Wesley, 1979) and my previous paper pp. 2-4.
See previous paper pp. 1-2.
Peter C.Ordeshook: Game Theory and Political Theory, an introduction , (Cambridge, UK: Cambridge University Press, 1986), 1.
The Usenet newsgroup devoted to the Diplomacy hobby, (rec.games.diplomacy) has seen nearly perpetual debate between those that would have cutthroat (standard) preference rankings be the only accepted norm for the play - charging that variation from this "spoils" the game - and those that assert that any objective that forwards enjoyment of the game is acceptable. Particular game masters of play-by-email games have disallowed draws in games where a stalemate position had not been achieved. Perhaps the inability to analyze others' behavior brought about by this phenomenon is what has frustrated the "cutthroats" - it certainly has frustrated this analyst/player.
If actions of the player effects outcomes without interaction with the actions of others, Game Theoretical analysis is not applicable. Rather, the decisions of a player are analyzable with the more general concepts of Rational Choice Theory.
Ordeshook, op. cit., 148.
lbid., 196-197
Andy Schwarz's variant, "Hundred," based on the Hundred Years' War, pits France against England and Burgundy.
Playing Diplomacy has given me practical experience in this matter. On one memorable occasion, after basing my diplomatic efforts and my tactical planning on the assumption that one of the other two remaining players would prefer to win than draw, and to draw rather than to be eliminated, I was eliminated forthright after failing (inexplicably it seemed to me at the time) by a two allies who happily enjoyed a draw over my dead body when either one could have achieved a full victory with stab of the other.

David C. Rosen
Central European University
([email protected])

If you wish to e-mail feedback on this article to the author, click on the letter above. If that does not work, feel free to use the "Dear DP..." mail interface.