Dr Frank Rosenblatt 1928 1971 n n Ph

Dr. Frank Rosenblatt, 1928 -1971 n n Ph. D Experimental Psychology, Cornell, 1956 Developed neural networks called perceptrons A probabilistic model for information storage and organization in the brain n Key properties n n n Association or learning Generalization to new patterns Distributed memory Biologically plausible brain model Cornell Aeronautical Lab (1957 -1959), Cornell (1960 -71) Frank Rosenblatt

Dr. Frank Rosenblatt n Wikipedia portrayal exaggerated “Rosenblatt was a colorful character at Cornell in the early 1960 s. A handsome bachelor, he drove a classic MGA sports car and was often seen with his cat named Tobermory. ” n Those who knew him would consider him a rather shy genius and more of a Renaissance man because he excelled in a wide variety of subjects, including psychology (his original field), computing, mathematics, neurophysiology, astronomy, and music Frank Rosenblatt

Agenda n n n n n The Mark I Perceptron – Visual System Model The Tobermory Perceptron – Auditory System Model Perceptron Computer Simulations Rosenblatt's Book Rosenblatt-Minsky Debates and Minsky-Papert Book Rat Brain Experiments Hobbies – Astronomy, Climbing, Music, Sailing Student Advocate Untimely Death Frank Rosenblatt

The Mark I Perceptron Visual system model and pattern classifier Examining A-unit of Mark I Typical three-layer perceptron: fixed S→A and variable A→R connections Frank Rosenblatt

The Mark I Perceptron Visual system model and pattern classifier n Input (sensory) layer of 400 photosensitive units in a 20 x 20 grid modeling a small retina n n Association layer of 512 units (stepping motors) each of which could take several excitatory and inhibitory inputs n n Connections from input to association layer altered through plug-board wiring, but once wired they were fixed for the duration of an experiment Connections from association to output layer were variable weights (motordriven potentiometers) adjusted through error-propagating training process Output (response) layer of 8 units Frank Rosenblatt

The Tobermory Perceptron Auditory system model and pattern classifier n Named after talking cat, Tobermory, in story by H. H. Munro (aka Saki) n Large machine n n n S-units: 45 band-pass filters and 80 difference detectors A-units: 1600 A 1 -units (20 time samples per detector) & 1000 A 2 -units R-units: 12, with 12, 000 adaptive weights A 2→R-units. Frank Rosenblatt

Perceptron Computer Simulations n n Hardware implementations made good demonstrations but software simulations were far more flexible In early 1960 s these computer simulations required machine language coding for speed and memory usage Simulation software package – user could specify the number of layers, the number of units per layer, type of connections between layers, etc. Computer time at Cornell and NYU Frank Rosenblatt

Rosenblatt's Book Principles of Neurodynamics, 1962 n n n Part I: historical review of brain modeling approaches, physiological and psychological considerations, and basic definitions and concepts of the perceptron approach Part II: three-layer, series-coupled perceptrons – mathematical underpinnings and experimental results Part III: multi-layer and cross-coupled perceptrons Part VI: back-coupled perceptrons Book used to teach an interdisciplinary course "Theory of Brain Mechanisms" that drew students from Cornell's Engineering and Liberal Arts colleges Frank Rosenblatt

Series-Coupled Perceptrons n n A perceptron is a network of sensory (S), association (A), and response (R) signal generating units A series-coupled perceptron is feed-forward S→A→R An elementary perceptron is a series-coupled perceptron with one R-unit connected to every A-unit and fixed S→A connections Convergence Theorem: Given elementary perceptron, stimulus world W, and any classification C(W) for which a solution exists, then if all stimuli in W re-occur in finite time, the error correction procedure will always find a solution Frank Rosenblatt

Series-Coupled Perceptrons Mark I was typical S→A→R perceptron Connections S→A: fixed, usually local A→R: adjustable w training Frank Rosenblatt

Series-Coupled Perceptrons A-units usually local biologically-plausible detectors Frank Rosenblatt

Series-Coupled Perceptrons Rosenblatt studied three and four-layer series-coupled perceptrons with two sets of variable weights but was unable to find a suitable training procedure like back-propagation Dotted lines are variable connections Frank Rosenblatt

Cross-Coupled Perceptrons n A cross-coupled perceptron is a system in which some connections join units of the same type (S, A, and/or R) Frank Rosenblatt

Back-Coupled Perceptrons n A back-coupled perceptron is a system with feedback paths from units located near the output end of the system to units closer to the sensory end Frank Rosenblatt

Rosenblatt-Minsky Debates and Minsky-Papert Book n n n Rosenblatt and Marvin Minsky (MIT) debated at conferences the value of biologically inspired computation, Rosenblatt arguing that his neural networks could do almost anything and Minsky countering that they could do little Minsky, wanting to decide the matter once and for all, collaborated with Seymour Papert and published a book in 1969, Perceptrons: An Introduction to Computational Geometry, where they asserted about perceptrons (page 4), "Most of this writing. . . is without scientific value. . . ” Minsky, although well aware that powerful perceptrons have multiple layers and Rosenblatt's basic feed-forward perceptrons have three layers, defined a perceptron as a two-layer machine that can handle only linearly separable problems and, for example, cannot solve the exclusive-OR problem Frank Rosenblatt

Minsky-Papert Book H. D. Block’s response paper n The authors address three classes of readers 1. 2. 3. n Computer scientists specializing in pattern recognition, learning machines, and threshold logic Abstract mathematicians interested in the debut of Computational Geometry Those interested in the general theory of computation leading to decisions based on the weight of partial evidence, e. g. psychologists and biologists H. D. Block concludes 1. 2. 3. Computer scientists “will find the book of little value” Abstract mathematicians consulted “were not captivated” “For psychologists and biologists, the level of mathematical maturity demanded will, I believe, make the book somewhat difficult to read. ” Frank Rosenblatt

Rat Brain Experiments n n n Late 1960 s – Rosenblatt began experiments in the Cornell Department of Entomology on the transfer of learned behavior via rat brain extracts Rats were taught discrimination tasks such as Ymaze and two-lever Skinner box, their brains extracted and injected into untrained rats that were then tested in the discrimination tasks to determine whether or not there was behavior transfer from the trained to the untrained rats Rosenblatt spent his last several years on this problem and showed convincingly that the initial reports of larger effects were wrong and that any memory transfer was at most very small Frank Rosenblatt

Hobbies – Astronomy Rosenblatt built a modest observatory on a hilltop behind his house 6 miles east of Ithaca. Work began summer 1961, Fecker 12" cassegrain telescope. He had interest in SETI (Search for Extraterrestrial Intelligence), wrote a proposal touting a "Stellar Coherometer” he designed, and was awarded $75 K for the project. Current photo – house in background The observatory was completed about 1966 – a circular cinderblock structure with dome housing the telescope Frank Rosenblatt

Hobbies – Music n n n Rosenblatt was an accomplished pianist and had a grand piano at his house in Brooktondale He played the well-known classical pieces of Mozart, Beethoven, etc. He also composed music and had a penchant of improvising endlessly on "Three Blind Mice" Frank Rosenblatt

Student Advocate n In 1969, the Afro-American Society occupied Willard Straight Hall, the student union, at Cornell University n n Recent video reviews the events Rosenblatt worked with other faculty to resolve the situation n This was later documented in a 1969 Interview with Frank n Frank gives a brief personal biography on pages 112 -116 Frank Rosenblatt

Practical Joker n As a graduate student Frank was a psychology major and Prof. James Gibson was a well-known faculty member and Frank’s dissertation advisor. As the story goes, Frank and some other graduate students drove to the town of Gibson one night and stole the town's "Gibson" signs, which they then mounted at the door of Professor Gibson's office. When Department Chair saw the signs, he remarked to the department secretary, "Don't you think Gibby's getting a little ostentatious? “ n Traveling to a conference Frank remarked “Do you think Mc. Culloch sleeps with his beard under or over the covers? ” Frank Rosenblatt

Rosenblatt – Renaissance man n n Excelled in a wide variety of subjects – psychology (his original field), computing, mathematics, neurophysiology, astronomy, and music He had two research reputations n neural networks with his perceptron work neurophysiology with the rat brain experiments When learning a new subject (to paraphrase Rodman Miller) n n n In a few weeks he knew a little In a few months he knew a great deal Soon thereafter he was discussing topics with experts in the field Frank Rosenblatt

Untimely Death Sailboat accident on his 43 rd birthday He “was a most gifted human being. . . had made his entire life a contribution to mankind” Congressional Record Frank Rosenblatt