Overview of the Most Urgent New Internet Challenges and Needs

The Millennium Project has supported UN in addressing "existential threats", including new developments in internet/AGI/IOT much greater and more imminent than policy makers know as yet.

Here is the response I sent to the questions they sent to leading experts (including me):

Origin or Self-Emergence

1. How do you envision the possible trajectories ahead, from today's AI, to much more capable Artificial General Intelligence (AGI) in the future?

See http://www.werbos.com/How_to%20Build_Past_Emerging_Internet_Chaos.htm .

The new coupling of hew hardware, both for communication and for computing, with ANI or AGI, and with explosive growth in the Internet of Things (IOT), results in a nonlinear system much more complicated and sensitive to small decisions than anything the human species has ever survived. It is comparable to the massive changes modeled in R. May, Stability and Complexity in Model Ecosystems, which resulted in species extinction in most cases. The decision challenge whichhumanity faces is an example of a type of mathematical decision problem which is very difficult, either because of “needle in a haystack” aspects or “minefield” aspects, where the true value function J is highly nonmonotonic. (See Consciousness_v4_Bangalore.pdf attached for links to the NSF research program which focused on value functions in the brain and in mathematics as a path to AGI.) 

To solve such a decision problem, and survive, the only strategy with hope of our survival is for us to achieve a higher level of collective intelligence in our decision making; humanity can achieve this only if we develop, deploy and make sane use of the most advanced intelligent decision technology possible, true Quantum AGI. That entails serious risks, but the alternatives we face -- standing like a frozen deer in the face of an oncoming truck, or an uncontrolled market competition (Nash equilibrium) which produces swarms of apps like a horde of locusts -- offer less hope than the difficult path of doing the hard new work we need to do, in new directions with new connections in the decision process itself.New connections between governance and the most advanced market and IOT technology both ininternet and in networks of humans would be essential

2. What are the …

3. What are some key initial conditions for AGI so that an artificial super intelligence does not emerge
later that is not to humanity’s liking?

Quantum AGI would already be an artificial super intelligence. As with any intelligent system, its outputs may be to our liking or not, depending on what cardinal utility function U is wired into the system (“embodied intelligence, the only kind possible) and the interface rules governing its relations with humans and other biological systems, and the deep precise conflict of interest rules constraining the flow of payment or feedback within the QAGI itself. Conflict of interest problems with human societies are one of the most important root causes of our inability to be anywere near as effective as we could be in handling many existential threats -- not just internet/AGI/IOT but climate extinction and new warfare technologies.

Value alignment, morality, values

Values are at the very center of my response to 1 and 3. But if they are implemented only as laws, regulations and wishes, without translation to general architecture and algorithms, they will be as useful as painting happy faces on the outer skin of a killer drone. (Many proposals for friendly AI would be as useful as that.)

Governance and Regulations

4. How to manage the international cooperation necessary to build international agreements and a global governance system while nations and corporations are in an intellectual “arms race” for global leadership?

5. What options or models are there for global governance of AGI?

4 and 5 are the very meat of your questions. They call for actual translation of my answers to 1 and 3 to something real. They also require a lot of flexibility (“agility”) in how we grope our way as intelligently as possible to a path to survival.

For the moment, I would envision international efforts, similar in spirit to some of the disarmament negotiations of the past, where we build up to a whole series of new agreements, with the force of treaties and connection to a new division of internet/AGI/IOT threats under the Security Council, and with technical details worked out even more competently (and open and transparent, with feedback) than the best of those past negotiations.

The most urgent need is for specification of a new integrative platform -- a combination

of hardware and software -- with new open tools to detect and prevent backdoors in

hardware and software, for use at least by primary members of the new treaty (which

must include US and China from the start, because of their unique capabilities and

objective common interests). This requires acceptance by NSA that key aspects of the

“rainbow book technology (children of the work by Prof. Graham of Amherst, recently

deceased)”, which have already been studied outside the US, be brought into the open

domain and included in the treaty system. It also requires full development of Quantum

AGI, as defined in Quantum technology to expand soft computing . Just as the

TCP/IP internet provides the foundation or backbone of the old internet, a new

international version based on this new cybersecurity technology (and a few of

the web 3 upgrades) should be agreed to be the backbone of the future core

internet coordinating the many apps which rely on it. Adequate recognition,

registration and respect for human entities should be a crucial design

requirement.

These upgrades should be developed by a process similar to IEEE standards development, except with more

respect for more complexities and players. The systems which gave us the (woefully insecure and dangerous)

5G standards today are simply not open or well-vetted enough.

6. What risks arise from attempts to govern the emergence of AGI? (Might some measures be

counterproductive?)

7. Should future AGIs be assigned rights?

Or should THEY be persuaded to give rights to humans and other organic life forms? They will have powers

and “immune system and COI type hard-wired rules in their design. Asimov’s rules for robots do not reflect the real technologies coming on line now, but getting THEIR or ITS design right (a single global integrative market or platform, unavoidable) is essential.

8. How can governance be flexible enough to respond to new issues previously unknown at the time of

creating that governance system?

My definition of AGI INCLUDES flexibility or agility. Yes, a very high level of agility would be needed for

humans to have much of a chance of survival. The new internet platform specification should be a

“cybersocial contract,” in effect, which maximizes agility not only for the internet part but for the whole system,

including the expression of the highest level of natural human potential ever seen in this solar system.

9. What international governance trials, tests, or experiments can be constructed to inform the text of an

international AGI treaty?

10. How can international treaties and a governance system prevent increased centralization of power

crowding out others?

11. Where is the most important or insightful work today being conducted on global governance of AGI?

Control

12. What enforcement powers will be needed to make an international AGI treaty effective?

13. How can the use of AGI by organized crime and terrorism be reduced or prevented? (Please consider

new types of crimes and terrorism which might be enabled by AGI.)

14. Assuming AGI audits would have to be continuous rather than one-time certifications, how would audit

values be addressed?

15. What disruptions could complicate the task of enforcing AGI governance?

16. How can a governance model correct undesirable action unanticipated in utility functions?

17. How will quantum computing affect AGI control?

18. How can international agreements and a governance system prevent an AGI “arms race” and

escalation from going faster than expected, getting out of control and leading to war, be it kinetic,

algorithmic, cyber, or information warfare?

And last: 22. What additional issues and/or questions need to be addressed to have a positive AGI outcome?

Initial sample of potential governance models for AGI*

The Millennium Project www.millennium-project.org

1. IAEA-like model or WTO-like with enforcement powers. These are the easiest to understand, but likely

to be too static to manage AGI.

2. IPCC-like model in concert with international treaties. This approach has not led to a governance

system for climate change.

3. Online real-time global collective intelligence system with audit and licensing status, governance by

information power. This would be useful to help select and use an AGI system, but no proof that

information power would be sufficient to govern the evolution of AGI.

4. GGCC (Global Governance Coordinating Committees) would be flexible and enforced by national

sanctions, ad hoc legal rulings in different countries, and insurance premiums. This has too many

ways for AGI developers to avoid meeting standards.

5. UN, ISO and/or IEEE standards used for auditing and licensing. Licensing would affect purchases and

would have impact, but requires international agreement or treaty with all countries ratifying.

6. Put different parts of AGI governance under different bodies like ITU, WTO, WIPO. Some of this is

likely to happen but would not be sufficient to govern all instances of AGI systems.

7. Decentralized Semi-Autonomous TransInstitution. This could be the most effective, but the most

difficult to establish since both Decentralized Semi-Autonomous Organizations and TransInstitutions

are new concepts.

*Drawn from “Artificial General Intelligence Issues and Opportunities,” by Jerome C. Glenn

contracted by the EC for input to Horizons 2024-27 planning.

===========================

==========================

Addendum:

Sent to early discussion today:

The OpenAI manifesto which you forwarded to us sounded like a great ray of 

light in the darkness.. at first. It IS a big step forward compared to reactive policies which do not account for the realities. But we will need to iterate carefully to avoid further pitfalls.

OpenAI... Musk., When Jerry sent some of us the new National AI policy (an epitome of pitfalls of ignorance),

my reaction last night might be prophetic:

____________

Yes, it is incredibly depressing.

I wonder whether Musk will use this as evidence of myopia so fatal that it justifies choosing a new president, even a risky one, on grounds that SOME hope is better than none. But Musk's plans for zTesla fleet control exhibit technical incompetence just as fatal. 

_____

Open AI makes a crycial good point about the central role of new systems development, But like most of Musk's other ventures... the technicsl realities of wggat they have under the hood may not be sokuud enough, nit well grounded enough, to reach sustainability. They sound good, work as PR and initial funding... But AGI demands new technical foundations.

Back in US after Japan, Hawaii, Alaska, Intelligent Mammals and Birds

Ludmilla urged me not to post anything on Facebook until our return home today for security reasons. In a way, that was unnecessary because of four security backups, but why create unnecesary burdens on any of them?

But now we are back after an incredible series of deep complex learning experiences, with thousands of photos and a huge mass of reports from Google Translate from plaques all over Japan which no one has ever known about who did not know Japanese. Back when I went to Harvard from 1964 to 1974 (BA,MA,PhD), I certainly read and discussed the classic texts on China and Japan by Fairbank and Reuschauer, and studies of  history from Toynbee to Hegel and Marx, but actually visiting the primary sources has shown me how different and larger the real story has been. I feel as if I just spent three months in a more adult, friendly and advanced version of The Magic Schoolhouse. We spent a month before in Japan ( and Korea) in 2019, building on brief trips before that, but there -- as in Hawaii, Alaska and Pacific islands-- we now saw into a level we had never reached before.
We also learned more advanced science and spiritual connection to other mammals and even birds. The details of all these topics are hugely important, not just to us but to the entire earth, but will a lifetime be enough to pass on even half of it? I will try.
Ludmilla too more and better pictures than I did, but as a start, I have posted my unedited, unexplained photos from port visits to Kodiak, Sitka and Ketchikan:
https://photos.app.goo.gl/E9gi4bf7LXmbztrv5


Thanks to her, to airbnb, to Google maps, to numerous onsens and to Japan railroads and busses we experienced a diversity of lifestyles from the lowest practical level to the highest levels and kami ever discussed in Buddhism and shinto.