 |
|
|
|
|
|
|
|
 |
|
||||||
 |
 |
 |
 |
 |
 |
 |
|
µ
Machine Understanding
More Tech Stuff:
Indexing Books: Lessons in Language Computations
Constructing a Mandelbrot Set Based Logo with Visual Basic.NET and Fireworks
April 9, 2010
Invariance Light
I've been thinking about the difference between my prior thoughts on invariance and how the word is used in Numenta's documentation. I became fascinated by invariance in the remote past. I probably first heard of it as a topic from studying tensor mathematics, with its applications to general relativity. I am not sure when I first started wondering how the human brain is able to call up images of objects (in the imagination) in a way that implies it holds invariant representations of those object in its memory. Reading On Intelligence by Jeff Hawkins reminded me of the issue a few years back.
I remember trying to explain my interest in invariance once to a non-technical person and realizing that she must either think it was a crazy idea, or just one of those tech discussions that was so far outside her knowledge base that I might as well have been speaking COBOL. The problem is that, under ordinary circumstances, there does not appear to be a problem at all. Ordinary solid objects are naturally invariant. My water bottle does not actually shrink when it is placed further from me, despite occupying a smaller angle in my visual field. To vary it I would need to break the glass or melt it.
Invariance only become a big deal in special circumstances. One is in physics, where the speed of light, gravity, and space-time are interwoven in such a way that special mathematical tools, tensors, are needed to correctly calculate results. Another is in psychology/neurobiology where the amazing ability to create 3D (really 4D, including time) memories and imaginings is critical to human intelligent behavior. People sometimes mistake dreams for reality, so capable is the brain of recreating, or creating, realistic scenarios.
But, as in all philosophy, science, and even ordinary life, we must not be deceived by a broad variety of related phenomena only having one word to represent them in English. There are differences between real-object invariance, the math allowing for invariant objects in physics, and the use of the term invariance when applied to the human brain and computer models of the brain.
In Bitworm we have a very simple (if you ignore the internal algorithms required) example of computer software modeling Hierarchical Temporal Memory (HTM), thought by the Numenta crew to be at least an approximation of how the human brain exhibits intelligence. And in a very simple sense, Bitworm constructs invariant representations of the sensory objects of its artificial world, bitworms. It can correctly categorize different types (species?) of bitworms.
The ability of neural networks, and some other types of artificial intelligence systems, to correctly identify objects, for instance human faces, is not new. Methods for recording objects and regurgitating the recordings goes to back to the 19th century. We also now have the capability to model objects in 3D and play with them on a computer screen, or in an artificial world like Second Life. All of this informs the use of the word "invariance" in neurology and machine understanding contexts.
I think it is worthwhile distinguishing shades of gray when in comes to invariance. The ultimate invariance is real-world, physical invariance. At the other end we have simple neural-like categorization systems of the HTM type. Once an HTM has learned to distinguish a few types of object, it has a combination of data and algorithms in its nodes that can be considered to be invariant representations of the objects. The outcomes could be matched to reproductions of the original data, but generally the HTM does not store the original data. The output of an HTM may be consistently correct about the cause of its sensory input; for now I'll just call that capability invariance light.
I believe that it is possible for some sort of network like an HTM to do what the brain does, at least: remember complex models of real world objects. As when I recognize a friend I see in the distance from the way she walks, and can remember, almost effortlessly, what her face looks like, close up, front view or profile view. Invariance for the brain goes beyond being able to recognize an object from a picture. We can cross senses. For instance, I can recognize ordinary objects by feel, then call up pictures of them in my imagination.
There is a mathematics to this capability. While the data may be spread out over HTM nodes, or various areas of the cortex of the human brain, it should still be representable with abstract data. Given the complexity involved, I think studying invariance light will be necessary if we are not to be confused by the complexity of systems approaching human capabilities of recognition and inference.
See also invariant (physics)