Reformulating Game Theory: The mathematics underlying human societies, conflict and cooperation

Basic Background Principles and an Important Example Previously, I posted a condensed overview of the basic concepts of telos, cardinal utility functions and the design of intelligent systems such as that part of the brain which manifests intelligence, true machine intelligence ("AGI"), and the "brain" part of noospheres. https://drpauljohn.blogspot.com/2020/02/neural-math-foundations-telos-purpose.html The mathematics of one intelligent system learning to maximize (the expected value of utility U over future time) is one of the two core foundations of neural network mathematics, and of "reinforcement learning" (e.g. systems like Alpha Go.). See http://springer.iq-technikum.de/referenceworkentry/10.1007/978-1-4471-5102-9_100096-1) for an up to date review of that area. GAME THEORY is the general theory of the strategies of action in a system which has N actors, N intelligent systems, each of which has its own utility function Ui ( i = 1 to N), its own choices of action and (in some variations) its own imperfect knowledge of the situation. Game theory was invented in the classic book by von Neumann and Morgenstern, the Theory of Games and Economic Behavior. This past week, my wife and son asked me to look at some current models they have for the interaction of political and economic forces in China, grounded in game theory. This reminded me that there are certain aspects of modern game theory which are essential to really understanding many of the issues we have discussed, from what Spengler discusses to what locke and Yeshua have said about social contracts. Some aspects have yet to appear in an integrated way in the literature of game theory even now. Here I will summarize what those pieces are, and some of the implications. (It clearly calls for new papers by folks who are mathematically competent, and I would of course be happy to support such work as a junior coauthor.) Principles of Extended Game Theory: Key Extensions of Von Neumann and Morgenstern The book by VonNeumann and Morgenstern addresses the case of STATIC games, where each actor i chooses his actions ui from a list of finite choices. The outcome (Ui for each actor i) is a function of the set of choices ui. This reminds me a bit of early work by computer scientists on 'reinforcement learning" where simple maximization of a static function U(x) was discussed as an example of "reinforcement learning." (See Neural Networks for Control, by Miller Sutton and Werbos, MIT Press, for how the communities thought about this in 1988.) Modern reinforcement learning, or RLADP, deals with decisions over TIME, as do brains. In that case, intelligent systems choose a vector of actions u(t) at each time t. To model the flow of human decisions in history (e.g. models of modern China), we need to assume the more general form of game theory in which each actor chooses ui(t) at each time t, and maximizes , the expected value of Ui(t) over future times t. in fact, there are THREE generalizations or extensions of Von Neumann and Morgenstern which are essential to understanding the kind of historical system which my wife and son are looking at now: (1) The theory of dynamical games. This is actually a huge set of fields by now. I usually cite the work by Bob Axelrod of U. Michigan, who is the best known thought leader of how to go from static games to dynamical games, and apply that to real social interactions. It has been applied to the issue of "how can we get to peace?", to strategies of action by the actors which lead to "better outcomes." ABSOLUTELY CENTRAL to this work is the effort to get closer to "Pareto optima." ROUGHLY SPEAKING --- a "Nash equilibrium" is what happens when each actor decides on his own, treating the other actors "like cardboard," like inanimate objects, and going for his own maximum in a myopic kind of way. a "Pareto optimum" is any one of the MANY possible choices which would be better for ALL the actors than a Nash equilibrium would be. (Some games exist where they are the same, but it is great challenge to design such games. The field of MARKET DESIGN addresses that challenge; the design of today's modern electric power system ("RTO/ISO") comes from serious mathematical efforts to address that challenge, and will be essential to building a sustainable internet of things and escaping the descent to chaos we now see in the new emerging global internet.) unfortunately, many people in engineering and internet design and modern military strategy think only in Nash terms; that THOUGHT is itself a major threat to our future history. When I think of those people, I think of the popular book about Nash, a Beautiful Mind, where the author is sadly oblivious to what Von Neumann was trying to say to Nash. (This is somewhat personal to me, after many trips to Princeton, but not for this post.) (2) The theory of "tact norms" in T.C. Schelling's easy-to-read classic book "Strategy of Conflict." This is just as fundamental as the theory of dynamic games itself!!! the theory of dynamic games reformulates what a "game" IS. Schelling's work explains aspects of COOPERATIVE Strategy, of how to get to a Pareto optimum, which are far beyond what Von neumann offered EVEN FOR THE STATIC CASE. (Of course, we need to integrate his insights into dynamic games, and apply to real human interactions. Schelling got deeper into those interactions even than the Axelrod family did, which is saying a lot. When I was a student of Schelling, he was frequently went to the telephone to advise Robert McNamara, then the secretary of Defense. His OTHER books connect to a lot of hard core even first person experience.) Von Neumann's book mainly uses the idea of COALITIONS as a way to find solutions for action in a many-player game. Though he assumed a static game, in principle, his idea of "coalitions" naturally fits REPEATED static games, and begins to shade into the world of dynamic games. It has been widely used in practical economics and political economy, where people do of course know about Von Neumann and Morgenstern. It is sad, then, that they do not know about Schelling. In a way, what Schelling argues(with LOTS of empirical examples, both from experimental games and international politics) is that games of mixed conflict and cooperation (or just cooperation) ARE NOT CLOSED. The mathematical definition of a game is simply NOT SUFFICIENT information to get us to a Pareto optimum, even when we know the dynamical rules as well. We already knew that the formal rules cannot tell us WHICH Pareto optimum to go for, but in fact that is at the core of the difficulty of real "games" of human history. Microeconomists sometimes play shell games trying to convince us that there is only one possible Pareto optimum, and thus that we should vote for it, but that IS a shell game, and is rarely one of the choices we get to vote for. Schelling basically argues that we need to have "tacit norms," a specific concept of POSSIBLE Pareto optima or norms of behavior which lead to them (or to outcomes better than than the Nash equilibrium; usually there is actually a "ladder" of possibilities up from Nash to better, partial Pareto eqiulibria to ideal never quite reached). Of course, the historical EVOLUTION and LEARNING of tacit norms is a crucial part of the longer term history of how humans learn to connect to each other even in the mundane world. The concept of learning has been applied in the past to dynamical games. Mathematically, it can be seen as en extension of the theory of learning by single actors, which is one of the many mathematical roots of modern neural network mathematics. FROM the theory of single actor neural networks (and its obvious extension here), we know that systems often fall into LOCAL MINIMA, which are often grossly suboptimal, like Nash equilibria. (Like war of all against all, which with autonomous weapons or nuclear weapons or both leads most easily and naturally to extinction of any species with our new technological capabilities. The efforts of Von Neumann himself were essential, in my view, to our not having gone extinct already, and it scares me that we have new generations of decision makers who do not really understand what he was telling them.) A key issue here is then: how can we FIND and COMMUNICATE ALTERNATIVE strategies, to get out of a local minimum, and find a discretely better basin of attraction? this is basically just the N-player extension of the challenge of local minima which some of us know inside-out after many decades of work in neural networks and in nonlinear dynamical systems. This is basically the same math which I used in discussions with Kozma and Freeman, in discussing "phase transitions" in brains and other systems. (It was also a major part of "chaos control" as studied by Yorke, Grebogi and Ott at the University of Maryland years ago, another group I have fond memories of.) This is the mathematical foundation necessary to really understand relations which many most sensitive and effective first person actors have learned at a subjective, intuitive level, essentially to the actual historical success of the groups of humans most successful over centuries in getting to better, more Pareto-like outcomes for their societies. What they learned is an example of what I call "first person science." Among the names which come to my own mind are Thomas Jefferson, Moses, George Washington, and Meng Tzu -- all creators of a CRUCIAL PART of what Spengler called "weltaunschaung," the mass of thought which makes possible large scale human civilizations, the erosion of which leads to collapse and death. (As today's world civilization is in danger of doing, to be ACCELERATED by what is happening on the internet unless we ALSO develop new tacit norms to undergird new social contracts between apps and hardware on internet). Jefferson, in particular, cited the work of Locke on "social contract" (as well as Francis Bacon and Newton) as the most important foundation of the new public school system which he promoted as an essengtial foundation to any hope of maintaning the more Pareto-like Republic he worked to help create. (It is painful for me not to say more about Washington, as it may even be his land I am typing on at this moment, but that is a topic for later, if anyone is interested.) But as Yeshua has often reminded us, the Ten Commandments and concept of Covenant from Moses are also so important as living memories in our noosphere, memories which fit the mathematics and really should be supported =, cultivated and extended. This is more or less just another was of talking about Max Weber's concept of "legitimacy"; indeed, Weber (a colleague of sorts of Spengler) expressed this concept somewhat more explicitly in its vast historical context. A proper presentation of the NEXT PHASE of general game theory, embodying tacit norms AND dynamics, should CONNECT that extension to actual historical dynamics (just as werbos.com/religions.htm BEGINS to connect OTHER parts of Splenger's weltaunschaung to one of the relevant PIECES of human history). (3) The work of Raiffa and its many uses I promised (3). ALSO important, and never to be forgotten or relegated to a footnote: the work by Howard Raiffa, who EXPLAINED a lot of Von Neumann's core concepts (from Von Neumann and MOrgenstern), and connected them to the realities of social dynamics IN INDUSTRY, IN THE ECONOMY. (Before information technology, I might even say I view DNA, money and the noosphere as the true underlying dynamics of human behavior through history. Well, I suppose that that was too informal. To ignore the economy is to be rather dumb and blind in the study of human history, and yes I also cite E.,O. Wilson, Robert May and Marx and Ken Arrow when I get into those details.) In the example of China, I often think of the incredibly important first person data we can share with Paulson (his essential book on Dealing with China) on how these systems really work, which of course fits nicely with work by Raiffa in other areas later in life. (He, not Trump, was perhaps the true expert on the real art of the deal, but in the end communication and marketing ARE important parts of the larger system, just as our toes and certain other organs are. I often think back to where trump actually learned SOMETHING about marketing and the middle "J" at Wharton... another weird overlap with my own first person experiences.) A Few Thoughts on by Response Received From Joshua Davis (Anglicized version of Yeshua ben David) After seeing this Yeshua also noted important work by Ron Howard of Stanford, whose work was also a crucial inspiration for the work on reinforcement learning, discussed in my previous post. Of course, when Yeshua says "we need a new social contract", I certainly agree. We really need to know what we are doing then, at a higher level than we did when facing less monumental social challenges, to have any chance of survival. We need to use everything we have. And, as that video from India (thanks to Varadan) said, that has ever been the way for groups of creatures trying to grow and survive in nature. Some Important Examples Taking the Basics Further: Sanity of The New Internet and the New China Extended game theory applies, of course, to all kinds of interactions of people and of "apps" (computer systems which live on larger platforms of hardware and of software rules). On the day when I copied over the text above from discussions with Yeshua, the cotonavirus crisis is stimulating people all over the world to discuss the huge changes already started all over the world. My previous note on Telos pointed to three new papers I published in 2019 explaining how the new neural network ADP mathematics tells us important thinks about the mental growth and experience of individual humans, and individual brains of all sorts, Ironically, you can find links to these three papers at werbos.com/religions.htm, which suggests that our entire noosphere can be understood as a single intelligent system, governed by one-actor mathematics! In the first part of the first paper, it discusses a very mundane starting point, essential to understanding maturity (stable convergence??) of a single intelligent system at the level we see in the human brain: the concept of "sanity" or "zhengqi." In my view, this concept is essential to really understand new possibilities both for China and for the Internet, In an INDIVIDUAL HUMAN BRAIN, we have a huge learned repertoire of how we use words in performing symbolic reasoning, in decideing what we believe and deciding in words what we decide to do. But the verbal part of the barin is just one layer of emergent things we have learned; it is not the foundation (which is non verbal) or is it the highest level of our symbiolic reasoning (which certainly includes mathematics). **IF** our symbolic reasoning is maure and coherent enough, we learn a few FUNDAMENTAL AXIOMS which we hold to; this is very much like the FIRST great work of Von Neumann, the axiomatization of set theory, (I am grateful to our local frined who showed us the readable and important book Turing's Cathedral, by George Dyson, which talks about Von Neumann's ligfe "in many worlds."). At a time in life, when the person is mature enough to make use of words in a powerful way, but when the AXIOMS learned for words are too weak or flawed in a dangerous way, conflicts arise. Indeed, this tends to fit Freud's picture of how nervous breakdowns happen. The key to survival and growth is to UPGRADE THE AXIOMS, and rise past the forces of reverting back to animal nonverbal behavior. But: "Ab below, so above." ENTIRE N-PLAYER systems like mundane human societies or collections of apps competing on the internet, CAN ALSO have this problem of a time of transition. In China, Jiang Zemin and his people represented a truly great advance in the "social norms," in the wletaunschaung underlying the society and economy of China. (Actually, Zhu Xi of an earlier dynasty played a similar role.) A beautiful, clear and powerful new synthesis, opening the door to growth and verbal consciousness. But in both cases, the new axioms had fatal gaps. In China, the attacks on QI were thge real cause, in my view, of the Xi JinPing antithesis. The crucial problem now is whether China will simply descend into chaos yet again (as it has many times in the past) by a process not ulike a nervous breakdown, or will it rise to a new synthesis, a new weltaunschaung? At this time, it looks bad, but there is hope if China does NOT GIVE UP ON SCIENCE, but ENHANCES it (both in learning and in social contracts), to be more like the original concept of science, which I discuss in those 2019 papers I just mentioned. FURTHERMORE... the new organizing axioms for the entire world (China included) WILL INCLUDE the new integrating platform used for both hardware and software of the world internet. (The main alternative would be somehow to destroy the internet, wihch mostly seems possible now only by violence so great it would probably kill all humans too. But any true expert on cosmos and history should consider Stapleton's fiction on that larger subject.) For a general overview of this challenge, see https://www.youtube.com/watch?v=6togqN9Cvt4. That talk discusses what the mathematics of intelligent systems tells us about what our choices are for the new integrating rules of the internet, which are an urgent matter of life or death right now. The movement towards a fatal Nash equilibrium in that game are very scary right now, scary than the scariest things people worry about as worst case possibilities for China. Futurists have told me that the FIRST PART of that video is not so relevant; indeed, it revews the powers of the US and EU today, whose solutions are indeed mainly irrelevant, to scary degree, but we do need to know where we are starting from. MUCH MORE relevant are the two greatest forces towards a new integration of the internet, which I call "Godzilla" and "King Kong." In essence, competition and incremental policy is moving us towards a top-down control system which does not provide the feedback and recurrence (let alone channels for the noosphere to express itself) necessary for real sustainability. There is also an emerging "Borg" power (really, I have seen those people!!)aimed at building "the new human" which they conceive of as a reliable puppet in the future top-down internet of things, compliantly doing what myopic humans at the top ask for. The novel Stealing Worlds by Karl Schroder gives a beautiful sense of reality on how unstable susch systems can be, but in fact we already have neural network tools which take it a step even beyond his novel. It is serious and urgent, and if we don’t get our ostrich heads up out of the dirt and face up to the need for a new ... EXPLICIT norm integrating the global internet... well, nature is full of "sink or swim" situations. =========== A few weeks ago, at the last Quaker Meeting I went to before the social distancing, many people meditated on the question: "IF God... or even just the noosphere,, is real, how could the world be falling apart as much as it seems to? Is this level of pain necessary?" And I remembered (with reminders from Yeshhua and Varadan) that even noospheres have parents, as best I understand. OUR LIVES are mainly in interaction with EACH OTHER, other parts of the noosphere, OUR duties and OUR nature as individual humans, body and soul. But in very special and unique circumstances, Jesus would ask us to call on our "Father who art in Heaven." When I asked for a bit of clarity on that... the image which came to me was: "Paul, you know what it means to be the father of a teenager. YOUR ENTIRE noosphere, from the sun to the depths of the earth and the Oort cloud, is one big teenage noosphere. Yes, teenagers go through growing pains. Some try to suppress the pain, with chemical or mental or electrical opiates, ALL FATAL. You MUST feel all that pai;, it is necessary to your growth, but you also need to become MORE AWARE, MORE CONSCIOUS (and yes, more scientific and mathematical) to get a grip on it all." And so the analogy to human personal sanity applies at all levels...