The Millennium Project has supported UN in addressing "existential threats", including new developments in internet/AGI/IOT much greater and more imminent than policy makers know as yet.
Here is the response I sent to the questions they sent to leading experts (including me):
Origin or Self-Emergence
1. How do you envision the possible trajectories ahead, from today's AI, to much more capable Artificial General Intelligence (AGI) in the future?
See http://www.werbos.com/How_to%20Build_Past_Emerging_Internet_Chaos.htm .
The new coupling of hew hardware, both for communication and for computing, with ANI or AGI, and with explosive growth in the Internet of Things (IOT), results in a nonlinear system much more complicated and sensitive to small decisions than anything the human species has ever survived. It is comparable to the massive changes modeled in R. May, Stability and Complexity in Model Ecosystems, which resulted in species extinction in most cases. The decision challenge whichhumanity faces is an example of a type of mathematical decision problem which is very difficult, either because of “needle in a haystack” aspects or “minefield” aspects, where the true value function J is highly nonmonotonic. (See Consciousness_v4_Bangalore.pdf attached for links to the NSF research program which focused on value functions in the brain and in mathematics as a path to AGI.)
To solve such a decision problem, and survive, the only strategy with hope of our survival is for us to achieve a higher level of collective intelligence in our decision making; humanity can achieve this only if we develop, deploy and make sane use of the most advanced intelligent decision technology possible, true Quantum AGI. That entails serious risks, but the alternatives we face -- standing like a frozen deer in the face of an oncoming truck, or an uncontrolled market competition (Nash equilibrium) which produces swarms of apps like a horde of locusts -- offer less hope than the difficult path of doing the hard new work we need to do, in new directions with new connections in the decision process itself.New connections between governance and the most advanced market and IOT technology both ininternet and in networks of humans would be essential
2. What are the …
3. What are some key initial conditions for AGI so that an artificial super intelligence does not emerge
later that is not to humanity’s liking?
Quantum AGI would already be an artificial super intelligence. As with any intelligent system, its outputs may be to our liking or not, depending on what cardinal utility function U is wired into the system (“embodied intelligence, the only kind possible) and the interface rules governing its relations with humans and other biological systems, and the deep precise conflict of interest rules constraining the flow of payment or feedback within the QAGI itself. Conflict of interest problems with human societies are one of the most important root causes of our inability to be anywere near as effective as we could be in handling many existential threats -- not just internet/AGI/IOT but climate extinction and new warfare technologies.
Value alignment, morality, values
Values are at the very center of my response to 1 and 3. But if they are implemented only as laws, regulations and wishes, without translation to general architecture and algorithms, they will be as useful as painting happy faces on the outer skin of a killer drone. (Many proposals for friendly AI would be as useful as that.)
Governance and Regulations
4. How to manage the international cooperation necessary to build international agreements and a global governance system while nations and corporations are in an intellectual “arms race” for global leadership?
5. What options or models are there for global governance of AGI?
4 and 5 are the very meat of your questions. They call for actual translation of my answers to 1 and 3 to something real. They also require a lot of flexibility (“agility”) in how we grope our way as intelligently as possible to a path to survival.
For the moment, I would envision international efforts, similar in spirit to some of the disarmament negotiations of the past, where we build up to a whole series of new agreements, with the force of treaties and connection to a new division of internet/AGI/IOT threats under the Security Council, and with technical details worked out even more competently (and open and transparent, with feedback) than the best of those past negotiations.
The most urgent need is for specification of a new integrative platform -- a combination
of hardware and software -- with new open tools to detect and prevent backdoors in
hardware and software, for use at least by primary members of the new treaty (which
must include US and China from the start, because of their unique capabilities and
objective common interests). This requires acceptance by NSA that key aspects of the
“rainbow book technology (children of the work by Prof. Graham of Amherst, recently
deceased)”, which have already been studied outside the US, be brought into the open
domain and included in the treaty system. It also requires full development of Quantum
AGI, as defined in Quantum technology to expand soft computing . Just as the
TCP/IP internet provides the foundation or backbone of the old internet, a new
international version based on this new cybersecurity technology (and a few of
the web 3 upgrades) should be agreed to be the backbone of the future core
internet coordinating the many apps which rely on it. Adequate recognition,
registration and respect for human entities should be a crucial design
requirement.
These upgrades should be developed by a process similar to IEEE standards development, except with more
respect for more complexities and players. The systems which gave us the (woefully insecure and dangerous)
5G standards today are simply not open or well-vetted enough.
6. What risks arise from attempts to govern the emergence of AGI? (Might some measures be
counterproductive?)
7. Should future AGIs be assigned rights?
Or should THEY be persuaded to give rights to humans and other organic life forms? They will have powers
and “immune system and COI type hard-wired rules in their design. Asimov’s rules for robots do not reflect the real technologies coming on line now, but getting THEIR or ITS design right (a single global integrative market or platform, unavoidable) is essential.
8. How can governance be flexible enough to respond to new issues previously unknown at the time of
creating that governance system?
My definition of AGI INCLUDES flexibility or agility. Yes, a very high level of agility would be needed for
humans to have much of a chance of survival. The new internet platform specification should be a
“cybersocial contract,” in effect, which maximizes agility not only for the internet part but for the whole system,
including the expression of the highest level of natural human potential ever seen in this solar system.
9. What international governance trials, tests, or experiments can be constructed to inform the text of an
international AGI treaty?
10. How can international treaties and a governance system prevent increased centralization of power
crowding out others?
11. Where is the most important or insightful work today being conducted on global governance of AGI?
Control
12. What enforcement powers will be needed to make an international AGI treaty effective?
13. How can the use of AGI by organized crime and terrorism be reduced or prevented? (Please consider
new types of crimes and terrorism which might be enabled by AGI.)
14. Assuming AGI audits would have to be continuous rather than one-time certifications, how would audit
values be addressed?
15. What disruptions could complicate the task of enforcing AGI governance?
16. How can a governance model correct undesirable action unanticipated in utility functions?
17. How will quantum computing affect AGI control?
18. How can international agreements and a governance system prevent an AGI “arms race” and
escalation from going faster than expected, getting out of control and leading to war, be it kinetic,
algorithmic, cyber, or information warfare?
And last: 22. What additional issues and/or questions need to be addressed to have a positive AGI outcome?
Initial sample of potential governance models for AGI*
The Millennium Project www.millennium-project.org
1. IAEA-like model or WTO-like with enforcement powers. These are the easiest to understand, but likely
to be too static to manage AGI.
2. IPCC-like model in concert with international treaties. This approach has not led to a governance
system for climate change.
3. Online real-time global collective intelligence system with audit and licensing status, governance by
information power. This would be useful to help select and use an AGI system, but no proof that
information power would be sufficient to govern the evolution of AGI.
4. GGCC (Global Governance Coordinating Committees) would be flexible and enforced by national
sanctions, ad hoc legal rulings in different countries, and insurance premiums. This has too many
ways for AGI developers to avoid meeting standards.
5. UN, ISO and/or IEEE standards used for auditing and licensing. Licensing would affect purchases and
would have impact, but requires international agreement or treaty with all countries ratifying.
6. Put different parts of AGI governance under different bodies like ITU, WTO, WIPO. Some of this is
likely to happen but would not be sufficient to govern all instances of AGI systems.
7. Decentralized Semi-Autonomous TransInstitution. This could be the most effective, but the most
difficult to establish since both Decentralized Semi-Autonomous Organizations and TransInstitutions
are new concepts.
*Drawn from “Artificial General Intelligence Issues and Opportunities,” by Jerome C. Glenn
contracted by the EC for input to Horizons 2024-27 planning.
===========================
==========================
Addendum:
Sent to early discussion today:
The OpenAI manifesto which you forwarded to us sounded like a great ray of
No comments:
Post a Comment