Could our brains, souls and effective governments all be based on neural net math?

At World Futures Day yesterday, a leading futurist in Germany asked me this question. My reply this morning: 

================================

Good morning!

It was a very great pleasure yesterday to hear from you, from Mila, from Vint Cerf  (the true father of the internet, now a VP at Google) and many other old friends in the World Future Day discussion. As Mila suggested... these kinds of discussions which grow into real spiritual connections are so important to our lives in so many ways!!!

You made  a brief comment which deserves a lot more followup. Maybe it is just as well that we did not have enough time yesterday!

In Mila's session, I quoted the old saying of engineers: "When your only tool is a hammer, the whole world starts to look like a collection of nails." And I went on: For almost 30 years, my main job was an engineer, but not the kind who makes hammers. I was the kind who makes brains... intelligent systems designed to do what brains do, fully informed by neuroscience and helping advance neuroscience itself. 

As of now, a few of us understand and have tested a whole new unification of neural network mathematics,which  not only fits what we see in the brain (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5125075/) but ALSO OFFERS important insights in how to design human organizations which work (e.g. see https://www.amazon.com/nerves-government-political-communication-control/dp/B0000CLW5Q/ as an early starting point) and in understanding our noosphere, in which we are like cells or groups of cells. 

It is universal and powerful mathematics, the best we can do in a cosmos governed by classical physics in the classical way.

=================================================

YOUR COMMENT:

Is it just about NETWORKS? What about field effects?

===========================================================

Please forgive -- but old men like us do have a right and even a duty to tell stories, especially of happy instructive memories.

You remind me of the ONE OTHER person who asked me exactly the same question, just as clearly as you did, with even more intensity.

This happened in the early 1990s, in Radford Virginia, when I was the new (third) President of the International Neural Network Society (INNS), introduced by my friend Sam to Karl Pribram, who was then one of the world's two leaders of systems neuroscience.See https://www.youtube.com/watch?v=nqF__KofTZE for some discussion of Karl and of Walter Freeman (the other great leader), 

and for some of the collaborations involving Yeshua and me. 

 

Karl was deeply offended by the Neural Doctrine as formulated by Stephen Grossberg (first President of INNS), the basis of all of Steve's models. THAT VERSION of neural networks assumed asynchronous processing - no clocks, governed by ordinary differential equations (ODE). It assumed that all the RELEVANT information flows go from the inputs to neurons (synapses) through a cell body governed by well-known membrane equations, on to outputs set to the axons (output cables) of cells. No backwards flows, no field effects. All learning based on changes in synapse strengths, which were "Hebbian" IN THE SENSE THAT the changes were all functions of LOCAL variables included in the forwards-moving calculations.

Karl was deeply worried that INNS might be another narrow rigid cult, like some of the modern Bohmian cults (where I do NOT include hard core realists as core members of those cults, even though they sometimes try to show up and remind them of "the first Bohm"). "That neuron doctrine simply does not fit what we really see in real empirical work." 

I was SO happy to be able to reassure him that no, INNS was not that kind of religious cult, that Steve very carefully organized it to be open and cross disciplinary, and even supported my nomination to be the new President after Widrow. (That happened when I left the room for a moment to go to the men's room, and was rather stunned when I came back.)  Backwards flows of information and clocks were absolutely central to Karl's view and to mine, as he was happy to learn. Yeshua and I have validated our theory of how the clocks work in the best hard data available when we did a decisive comparison https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5125075/.

Regarding backwards flows... Karl wrote an endorsement for my book, giving the original papers on backpropagation,

at https://www.amazon.com/Roots-Backpropagation-Derivatives-Forecasting-Communications/dp/0471598976/. On that web site you can look inside, and see exactly what Karl wrote. It is an important part of understanding how Karl really looked at things. 

BUT WHAT ABOUT THAT THIRD AXIOM OF STEVE'S DOCTRINE, THE ISSUE OF FIELD EFFECTS?

Karl Pribram and Walter Freeman were both VERY strong proponents of the importance of field effects in the brain.

I hope, Heiner, that you read this far, and are delighted to learn this.

People who really follow Pribram or Freeman would also ask what you asked. Karl's Magnum Opus, Brain and Perception, certainly talks about field effects a lot. Kar and I also had long discussions. And Walter's Magnum Opus, the book by Freeman and Kkozma, also has MANY diverse views represented on that.  (Including Vitiello, Baars, and me... not identical, though I generally agree with Bernie on the big picture.) 

But what KIND of field effects, HOW?

Karl explained, both in his book and in what he said, that he was most impressed by electromagnetic effects (which by the way is what holography is about in most human technology) up in the dendrites of neurons and even connecting groups of large neurons like the giant pyramid cells which he and I both revere(d). THESE ARE THE REAL GREAT PYRAMIDS WHICH WE NEED TO REVERE! The ones in Egypt are but shadows of this great real thing, present in YOUR neocortex! (And, I claim, in the nervous system of our noosphere as well.) 

"Those nonlinear cell bodies and axons are just the READOUTS, truncating and transmitting the IMPORTANT information, calculated by a vast linear holographic system, not unlike Hopfield networks, linear but very very complicated." 

Karl and I were invited to lead a special session in the IEEE Systems, Man and Cybernetics (SMC) world conference, held that year in Chicago. I thought I gave two talks in that session, but unfortunately I quickly find only ONE of the two (attached). Karl's is listed at

https://ieeexplore.ieee.org/xpl/conhome/665/proceeding,  as https://ieeexplore.ieee.org/document/271800. In that talk, and (I think) in my chapter in Karl's edited book on rethinking quantum theory 

https://www.amazon.com/Rethinking-Neural-Networks-Biological-Proceedings-ebook/dp/B00JKF0B9G/, I proposed that Karl's view of those field effects could be modelled by showing how Hopfield like linear field effects could be used as part of defining (and training) a piecewise linear neuron (or set of neurons), far more powerful than the standard linear model. I think Karl agreed with me that this captured his view of these effects and what they mean for the brain far better than the usual field ideas which were either "in the  mud or lost in the clouds." Back at NSF, I even funded a proposal by Todd Leen (NOW AT NSF HIMSELF!!) to develop a kind of piecewise linear neuron model which I felt might open the door to this additional degree of possible power.

Why have I done virtually nothing with a model/design which might offer orders of magnitude more power than the simpler types of neural network model popular today, even in most of my own work? 

ONE REASON: As one person, connected to HUNDREDS of life or death unmet opportunities, I find it hard to budget my time. I must prioritize. And hope that Todd himself might appreciate the thread I viewed him as our best hope for, which could indeed feed into many other threads (some of which he too may be putting time into now). 

Another -- when we get to hardware, there are OTHER ways we can implement this kind of higher order power which I sometimes try  to advance, either for new advanced dedicated neural network hardware and for new types of quantum neural network hardware

(as in https://www.sciencedirect.com/science/article/pii/S2772941922000011 and in extensive supporting material). I would be happy to discuss further, because these are important threads.

Yesterday, Vint asked about neural network designs which could handle issues like object recognition (and decision systems taking advantage of object identity). I will not repeat that long discussion here -- but here is a relevant story. In 2014, in a plenary talk at World Conference on Computational Intelligence (Beijing), I presented a ROADMAP of neural network designs (grounded in good old classical physics, with a caveat or two) https://arxiv.org/abs/1404.0554. Object identity, rising up from old style simple networks to convolutional neural networks to CSRN and Object Nets and beyond, was all reviewed as part of STAGE 2 of that roadmap. So that roadmap gave pointers and citations on how to solve the kind of problem he discussed.