IEEE Conference on Big Data
From Health to Brains to Stars
IEEE is the world’s largest
professional society in engineering, with more than 400,000 members all over
the world. Most of its societies, like the Computational Intelligence Society
(CIS), have a significant surplus of cash from their conferences, journals, and
magazines. Last year, to “give back” some of this money to the community, CIS
held a competition to fund ideas for summer schools or conferences. That is how
they funded a “summer school” (a large workshop) last week on Big Data in
Computational Intelligence: From Basic
Principles to LargeScale Applications. The conference was mainly made up of
seven 1.5hour talks spaced over two days, illustrating the key role that big
data and computational intelligence tools can play on areas from health care
and environment to brains and stars. I gave the first of these, bringing
together the big picture both on risks and opportunities and on the underlying
mathematics. The organizer, Nian Zhang of the University of the District of
Columbia, plans to post the talks on the web, and had important assistance from
others in the community. The talks and speakers, coming from great distances, are described at:
http://www.udc.edu/winter_school/IEEE_Cis_winter_school.htm
Here I will not post my own
slides (even though they were quite colorful and took a lot of work).
In fact – I apologize to you.
UNLIKE most of my blog posts, this is NOT written to be understandable to
anyone except me. These are just my notes to myself after the conference, and
preparing for a seminar I will give in Memphis. I usually do not post my
personal journal entries, but ... just once, it should do no harm, I hope.
=======================================================
=========================================================
The UDC conference last
weekend and the coming Memphis challenge have both been very stimulating to my
thoughts – enough to remind me what I have been missing since retirement, and
to remind me it would have been better if I had always taken the time to write
down the insights I get before I forget them. Robert Kozma’s talk at UDC was
“from brain to stars,” and indeed the new thoughts out of my usual daily
routine mainly come in those two areas.
But: a lot of it builds on
previous hard thinking here at home (easier than at NSF as schedules and
culture get more and more hectic), and on the challenge ahead. Bit by bit, I
feel called to broaden my attitude on about three things (as noted on page 87
of X2015, which I wrote in “like the old days” yesterday, sitting in a
comfortable wooden chair with cushions tilted back, in this study, with new age
music – Mannheim Steamroller and Kitaro – playing in the background, through
this computer): on brains/minds, to get past “point prototypes” and the early
syncretism model, to a fullfledged appreciation of Jungian things from math to
noosphere; from Z to richer structure of time tracks (evaluating data of
course); and – more today in particular, related to seminar to be given at
Memphis – richer understanding of emergent behavior of nonlinear dynamical
systems.
To some extent, my core job
at Memphis is just to present what I have already done. The challenge is to
make it comprehensible. Yesterday and in bed this morning, I tried to do more,
to deepen the understanding... but because of limited time, I need to just
record the basics of the new stuff and the new questions (incomplete as it is) this
morning, and then freeze it, and move on to preparing talk (and reading Yury’s
thesis).
Luda this morning reminded me
that I need to be very careful with HOW I define my terms for Memphis. For
example, I can take time with definitions, so long as I use the right language.
Functions, distributions and measures are OK, but I need to be careful not to
use the word “field” in a way which conjures up terror. But there is the
advantage that I can say “look at this small special case” and not worry about
people noticing I have included the whole universe and a highly
multidimensional continuum of its relatives.
Are PDE dynamical systems? Sometimes yes, sometimes no. Dynamical
systems are allowed objects, and I guess even definable. In fact, I do need to
review a lot of ODE (and/or Prigogine space) objects to put it into context
anyway, from Mehta to Boltzmann like.
But in the green notebook
yesterday... there were ideas I also need to get down on computer, for two
reasons: (1) to consolidate the logic a bit; and (2) to make it readable by me,
with the major issue of eyestrain now permanent after cataract surgery. (A big
ugly red hemmorage removed any doubt that I must be careful now as long as I
live!)
My initial goal (when I sent
abstract for seminar to Robert) was to explain the new generalized Boltzmann
distribution (in a couple of forms) given in my “extended GlauberSudarshan”
paper in arxiv. presented in 2014 in Scully’s workshop.
In practice, the consequences
are almost the same as those reported in my much older condmat paper: for
continuous states S(t) a probability Pr(S(t)) which is just f(E, C_{1},
... C_{m})dS(t), where f is some function of energy and of the m
conserved quantities of the theory. (m=0 is an allowed possibility, but the
approach breaks down when there is no conserved energy.) This differs from the
old idea that “the universe must go to a disorderly heat bath” (which Harold
Szu reiterated at UDC, referring to a debate between Boltzmann and Poincare) because
in the PDE case the derivative terms in the energy function do insert some
connection/correlation between neighboring points, at infinitesimal separation.
Still, that seems to suggest, on the surface, that the emerging order (pdf not
wellapproximated by the independence assumption across points in space) is
mainly at a small scale.
Yet... I feel called to try
to stretch further and be more open. So when I read Modesitt’s new novel, The
Solar Express, I did take notice of his discussion of systems where there may
be more pattern/regularity/order at LARGER distances than at smaller ones. (And
of course for years and years I have remembered a paper by Wheeler, where he talks
about neutrino temperature and tweaks things to account for the glaring fact
that there are these objects called planets and stars, rather fundamental to
what we see.) How does that work? Can that kind of patterning be
understood/explained within the confines of firstorder Hamiltonian systems,
using the energy term f and not the invariant measure dS as a source of rich
emergent behavior? Precisely how do open systems change things – remembering
that there is such a thing as classical chaos in conservative systems (was that
chaotic threearm broom balancer due to Poincare?...)... and that the universe
as a whole is presumed to be a closed system? (e.g. If chaos can occur in
Hamiltonian systems, why not fractal patterns?)
First observation: I have
ALREADY looked into emergent order, without naming it as such! The Boltzmann
distribution is a function of conserved charges C_{k}, as well as E. It
is precisely one of those conserved charges, the topological charge(s), which
leads to that very important example of patterning or order, the topological
soliton – i.e. the true elementary particles, the very foundation of everything
we see on earth! Even more interesting... albeit just at vixra and my followon
internal journal entry... “local minima” in the classical energy function also
lie behind the various metastable states of atoms, as was worked out in some
detail there! So YES, the Boltzmann term can do it, can generate “macroscopic”
order on distance scales as huge as the Angstrom unit, so many orders of magnitude
larger than the femtosecond scale we start with for the elementary particles.
Two quantum jumps to greater distance already, just based on f(E,C_{1},...C_{m})!
Beyond that, what comes next are solid objects, things I know well from quantum
electronics and photonics, and they certainly have interesting Boltzmannbased
patterning varieties. I remembered Roger Lake... how the same topological
soliton kind of pattern distribution can be elevated to the ordinary laboratory
level... and the quantum separator (QS) at that scale. (Not to mention some of
Von Neumann’s ideas about life.) So already with closed PDE systems (DO I SAY
“CLOSED” OR “CONSERVATIVE” AT MEMPHIS???)... the energy term DOES allow all
kinds of neat stuff. This week, I seem to be supporting Poincare as much as Von
Neumann... maybe I need to look him up more... later... (though for Memphis, it might help to show
links...)
Now of course, the patterning
we see with life on the surface of the earth (and the neuron example in the
Solar Express,,, which I probably should WRITE DOWN when next I visit Central
Library)... is due more to the injection of forwards time free energy, light
from the sun. But what specific kinds of boundary conditions count as “forwards
time free energy”? It is embarrassing how elusive it is to try to pin that
down. In a way, we may not really NEED to formalize the GENERAL idea; we need
Pr^{} and Pr^{+} boundary conditions at final and initial
times, and that’s enough; more may not even be inherently welldefined, as it
is ultimately just a rough way to characterize emergent statistics and specific
local experiments/systems. But if we can see something more clear and more
universal, it could be nice...
As I think more about that...
I am intrigued by the possibility of getting more concrete (more specific,
useful) results form a couple of special cases of “open” systems, which I think
of as the “uniform light” and “cosmic ecology” examples/cases/paradigms. In the
“light” example, we still have PDE, which can be represented by spatially
uniform additional terms violating timesymmetry and/or energy conservation.
(Now that I think of it... we have not proven that uniformity in 3D space would
work... a vast ocean of light model ... in generating life as well as our local
light does, entering in 2D. So many issues to explore!) Can we get nice
equations for that? The cosmic ecology would start more concretely...
considering things like rates of stuff ejected from black holes, gross
recycling of stuff keeping the universe alive, described as close to ODE as
possible... trying to keep it as simple as possible but real. (That reminds me
of very important new supernova results from Kozma/Chile, which we will pursue
after Memphis and after a brain exercise if all goes well.)
Thinking about the “light”
example... it is striking how my previous results (in “Extended..”) really did
rely on energy conservation. Yet shouldn’t the concept of equilibrium
probability be expressable in a USEFUL closedform way for density operators in
the general case? In my mind... I think of how the basic Hamiltonian dynamics
translate into dynamics for rho, with an obvious equilibrium condition. But
isn’t that condition in the GENERAL case (e.g. including light systems) about
as useless as Wiener’s nonlinear statistics, and in fact quite similar? It
reminds me of the inherent problems in reification, and the inherent stuff in
Arnold and in chaos.
So: for later. Something
probably is there... some ordering... but not this week.
============================
brainy stuff:
Perhaps I should also type
some of my thoughts about brain/mind/NN which came during UDC. Much fuzzier,
and not part of an active goal beyond my own mind... but... whatever.
I was disappointed, of
course, that Polikar – who seemed on course to solidify and upgrade syncretism,
to address the crucial issue of realtime learning in brains – has drifted away
into drifting away. A lot of the questions his new community are asking about
nonstationarity are no more puzzling than causality, and perhaps I should have
written some of it down earlier. (I have always had too much stuff to be able
to publish so much as half of it.) Yet he has some stimulating questions, if
the right questions are pulled out and played with.
For example: he asked (with
some reification here): “What if we have two sets of variables, Y(t) and X(t),
called labels and data? What if we have some initial data on Y(t) and X(t), and
then a WHOLE LOT more data on X(t) later, especially in a world where relationships
may change?” I urged consideration of two examples, an “auto” example and a
linear example. In the auto example... Ford might have millions of data points
on Y(t) and X(t) from thousands of engines or batteries, fully instrumented in
the lab, followed by later getting billions of data points just from X(t) from
cars on the road. (Hypothetical, assume wireless “phone home.”) The obvious
approach is to learn just from the laboratory data, but also learn soft sensing
to predict Y(t) from history of X(t) for use on the road. (Feldkamp and
Prokhorov have an important soft sensing paper in Narendra, which I even repost
on my own www.werbos.com/Mind.htm.) Later X(t) doesn’t do much! HOWEVER: in principle,
one could use X(t) (predicting X(t+1) from X(<=t)) to find proper “hidden
nodes,” preferred encoding, which one could then go back and use to REMODEL the
laboratory data. ALSO: observing the range of variation of the later data, IF
it is larger in some dimensions than the laboratory data, one can use it to put
special weight on trying to get accurate measurement of weights which will
explain more of the variance in later observations, a possibility for value
weighting. (Still, a concept much trickier than Vapnik understands.) The linear
example reminds us... if covariance of X to X changes, why not X to Y.... which
rather grossly invalidates that kind of exercise.

I worked hard, in preparing
my own talk (recorded by UDC, to be posted), to try to avoid being as
depressing as I have been lately at home. Probably death of the entire species
really is depressing, more so than my cancer, which still bugs me as well. I
did not mention climate change hardly at all. But Diego, the final speaker on
the first day, got into great detail on the kinds of stratospheric ozone
effects which, after H2S upwelling from oceans, seem likely to get worse enough
and be the first thing directly fatal to human life from climate as such. He
also did a lot of mixture of experts modeling, though he did not remind us of
what is or is not known about that general power of such things. Is there any
such? I remain skeptical, and skeptical of its relevance to brains. Yet I must
admit... that Jungian archetypes are beyond the point prototypes of classical
syncretism, and that mixture of models thinking might be adaptable to that
general case in a reasonable way, for noospheres if not for mammal brains.
Worth thinking about, along with dynamics and drift of prototypes in general.
The final talk by Kozma
included example of a solar surface model by Catalina Terra. (Met her to be
sure!) Amusing what degree of parallel with Modesitt’s story... different
desert, but still...
Wunsch objected to use of PSO
to optimize, a comment I certainly found friendly. For general power, we need
to get past methods like PSO which do not offer a good ratio of
parametershandled to number of examples explored... yes, use derivative and
hence the chain rule for ordered derivatives... but when a dozen or two free
parameters can do the job, priority in becoming more scientific involves two
different upgrades, as I said to people: (1) technical aspects, related to what
they are modeling now; and (2) new questions, the most important.
Re (1) :”Did you train and
test?” It is remarkable that one PDE type model (even if with nonlocal field
effects) could start form the beginning and track the long history of light
emission from any star without new data coming in. (Or did it? Train and test
needed.) I have seen cases in chemical engineering (my papers with McAvoy)
where such was possible, but others in economics (my 1977/8 SMC paper
predicting conflict in Latin America) where it was not, because of something...
a general form... of “phase drift,” shown in my slides for UDC. UNLIKE the
“nonstationarity” community... we have a compromise method which can address
this kind of thing... one more thing
which needs to be translated even now... and a fundamental thing at that! (Some
of the other things Polikar talked about can be modelled simply as hidden
variables in timeseries model, no fundamental significance at all, except
insofar as some parameter values need to be thought about...
part of the universal vector
intelligence game). Even now, I guess I need to be even more brutally explicit
to the world in explaining/defining the basic mathematical concept of vector
intelligence.
========================
From UDC, other loose ends...
tell Lily about MomLink, Maia about Harold’s earthquake model (communicated and
translated better) and Wunsch’s effort to use ADP on iterated prisoner’s
dilemma (and his hypothesis about decomposing some types of games into “easy”
and “PDlike”.) And Chawla’s health systems... maybe even related to my health!
Luis for political cooperation? Whatever... if time allows.. I did look up
Gulen, and need to explore loose ends in that area.
Antiaging also came up, at
UDC and in Solar Express, at many levels, from structures to people to ,...
well, I always think about noospheres too. It too relates to the Boltzmann
world, but has to be managed carefully in the weird environment we are in
today.