Logic in action A centipede was happy quite

Logic in action A centipede was happy, quite, Until a toad in fun Said, "Pray, which leg moves after which? " Which raised her doubts to such a pitch, She fell exhausted in the ditch, Not knowing how to run. -Sir Edwin Ray Lankester, 1898

Two perspectives on perception Hermann von Helmholtz (1821 -1894) Perception is “unconscious inference” James Gibson (1904 -1979) Perception is Direct: “the information is in the light”

Helmholtz on Perception Our sensations are simply effects which are produced in our organs by objective causes. . . [O]ur sensations are signs, not images, of such characteristics. . [However], even if in their qualities our sensations are only signs whose specific nature depends completely upon our make-up or organisation, they are not to be discarded as empty appearances. They are still signs of something - something existing or something taking place - and given them we can determine the laws of these objects or these events. . [T]he connections of ideas which take place in these processes [are] unconscious inferences. These inferences are unconscious insofar as their major premise is not necessarily expressed in the form of a proposition; it is formed from a series of experiences whose individual members have entered consciousness only in the form of sense impressions which have long since disappeared from memory. Some fresh sense impression forms the minor premise, to which the rule impressed upon us by previous observations is applied. Hermann von Helmholtz, ‘The Facts of Perception” (1878)

Gibson on perception: It is not necessary to believe that anything whatever is transmitted along the optic nerve in the activity of perception. . . We can think of vision as a perceptual system, the brain being simply part of the system. The eye is also part of the system, since retinal inputs lead to occular adjustments and then to altered retinal inputs, and so on. The process is circular, not a one-way transmission. . . The eye-head-brain-body system registers the invariants in the structure of ambient light. James Gibson, “The Ecological Approach to Visual Perception” (1979) [S]timulus information can specify its source in the world without necessarily being like it. James Gibson, “A Further Note on the Perception of the Motion of Objects as Related to the Perception of Events” (1966) The information is in the light.

Another description of the difference: Helmholtz: perception is like scientific discovery Gibson: perception is like bacterial chemotaxis The direct perception debate has a religious character, reminiscent of arguments about points of doctrine like free will and predestination. Aside: the role of inference in perception is connected to a philosophical argument about morality. . .

David Hume (1711 -1776) Right and wrong are like sense impressions, so that morality is “more properly felt than judg’d of”.

Our decisions concerning moral rectitude and depravity are evidently perceptions; and as all perceptions are either impressions or ideas, the exclusion of the one is a convincing argument for the other. Morality, therefore, is more properly felt than judg'd of; tho' this feeling or sentiment is commonly so soft and gentle, that we are apt to confound it with an idea, according to our common custom of taking all things for the same, which have any near resemblance to each other. (emphasis added) If you assert, that vice and virtue consist in relations susceptible of certainty and demonstration, . . . , you run into absurdities, from which you will never be able to extricate yourself. David Hume. A Treatise of Human Nature, book III, “Of Morals” (1740)

Thomas Jefferson disagreed with Hume’s political slant: Every one knows that judicious matter and charms of style have rendered Hume's history the manual of every student. I remember well the enthusiasm with which I devoured it when young, and the length of time, the research and reflection which were necessary to eradicate the poison it had instilled into my mind. . It is this book which has undermined the free principles of the English government, . . . , and has spread universal toryism over the land. -Thomas Jefferson letter to William Duane Monticello, August 12, 1810

So who’s right? Is perception (of form, color, motion, pitch, speech. . . ) ‘feeling’? or ‘judging’? Maybe both? Gibson never denied that there was complicated processing going on -- he simply wasn't interested in it. Gibson carried a hearing aid in his shirt pocket, with a prominent volume control. If you started talking about mechanisms, he would simply turn off his hearing aid, and that would be the end of that. For him, finding out what in the environment made perception possible was more than enough challenge for a lifetime. . . Bruce Bridgeman, “Simple Conscious Percepts Require Complex Unconscious Processing”. Psyche (1998)

Marr to the rescue • David Marr (1945 -1980): – Neuroscientist who worked at MIT AI Lab – Inventor of computational neuroscience • as a field • as a mode of study • “Vision” (published posthumously in 1982): Any information-processing activity can be described and investigated at three levels of increasing abstraction: – implementation: operation of the hardware – algorithm: design of the software – computation: analysis of the problem being solved

An example from Marr Implementation: – “Center-surround” or “Mexican hat” receptive fields of neurons in retinal ganglion cells – previously discovered by physiologists, but the function was unclear What is the computation being performed? What algorithm is the “Mexican hat” receptive field part of?

• Implementation: “Center-surround” or “Mexican hat” receptive fields of neurons in retinal ganglion cells • Algorithm – synaptic weights + cellular summation = inner product – response of each “center-surround” cell is the inner product of the image with “Mexican hat” pattern • at a particular place in the image • at a particular spatial scale (size of receptive field) – collective response of a population of similar cells equals convolution of the retinal image with a 2 -D pattern • Computation – a linear filter for sharpening edges in the image – similar image processing methods used in scanners, copy machines, photoshop – Math: “Laplacian of a 2 -D Gaussian” • Laplacian gives 2 -D derivative • Gaussian provides smoothing at a particular scale

2 -dimensional Laplacian-of-Gaussian (Lo. G) filter

Application of 2 D Laplacian-of-Gaussian filter to an image

Marr: retinal ganglion cell responses sharpening up edges in the image Is this part of a Helmholtzian process of perceptual inference? or part of a Gibsonian process of direct perception? Are the edges “in the light”? Does edge detection create hypotheses that can be true or false?

If perception is unconscious inference – what kind of inference is it?

The “-duction” product line: DEDUCTION: reasoning from the general to the specific All men are mortal. Socrates is a man. Therefore, Socrates is mortal. INDUCTION: reasoning from the specific to the general [telephone survey of 1, 007 Californians] 40% of California voters favor Arnold (± 3%) ABDUCTION: reasoning from observations to the best explanation [. . . detective work. . . ] The butler did it.

Helmholtz updated by Shannon • Task: available signals → model of the world – signals are mostly accidental, inadequate – sometimes disguised or falsified – always mixed-up and ambiguous • Reasoning about signals: – Context: what do you expect? – “Sensor fusion”: vision, sound, smell etc. – Source separation: there’s more than one thing out there – Variation: perspective, sources • Much harder than chess or calculus!

Claude Shannon (1916 -2001) Invented: application of Boolean algebra to logic circuits (1946) information theory (1949) – relation of messages and signals “. . . known for keeping to himself by day and riding his unicycle down the halls at night. . . ”

Bayes again • Thomas Bayes (1702 -1761) – Minister of the Presbyterian Chapel at Tunbridge Wells – Amateur mathematician – Essay towards solving a problem in the doctrine of chances, published (posthumously) in 1764 • Crucial idea: background (prior) knowledge about the plausibility of different theories can be combined with knowledge about the relation of theories to evidence • • in a mathematically well-defined way even if all knowledge is uncertain to reason about the most likely explanation of the available evidence Bayes’ theorem – – – “the most important equation in the history of mathematics” (? ) a simple consequence of basic definitions, or a still-controversial recipe for the probability of alternative causes for a given event, or the implicit foundation of human reasoning a general framework for solving the problems of perception

Helmholtz on unconscious inference in perception: [The] major premise. . . is formed from a series of experiences whose individual members have entered consciousness only in the form of sense impressions which have long since disappeared from memory. Some fresh sense impression forms the minor premise, to which the rule impressed upon us by previous observations is applied. Helmholtz is using the language of the “syllogism”, which is a model of deduction going back to Aristotle: Major premise: All X are Y Minor premise: Z is X Deduction: Therefore, Z is Y Is Bayes’ rule a form of inference? which kind?

Another component of Bayes’ analysis of “a problem in the doctrine of chances”: --costs and benefits of the alternatives = “utility function” Combined with prior, likelihood to “place your bets” on alternative choices of action. . .

Weiss, Simoncelli & Adelson • “Ideal observer”: artificial Bayesian model for 2 -D velocity estimation • Assumptions of the model: – – (non-linear gain control for perceived contrast) Changes in image intensity are due to translational motion Local image measurements are noisy Image velocities tend to be slow (“choose the slowest velocity estimate consistent with the data”) • Stimuli: Moving rhombus, moving grating, moving “plaid” • Method: Compare human experiments with – Bayesian model – Three alternative theories • Intersection of constraints • Vector average • Feature tracking

(a) Two drifting gratings superimposed in the image plane produce a translating ‘plaid’ pattern. (b) Dotted lines indicate constraint lines; arrows indicate perceived direction of grating viewed in isolation. The Interaction of Constraints solution (circle) is the unique velocity consistent with the constraint lines of both gratings. The Vector Average solution (square) is the average of the two normal velocities.

(a) Narrow rhombus at high contrast appears to move horizontally (consistent with IOC). (b) Narrow rhombus at low contrast appears to move diagonally (consistent with VA). (d, e) Fat rhombus at low or high contrast appears to move horizontally (consistent with IOC).

Effect of contrast and noise Model’s likelihood functions for three local patches of a horizontally translating diamond stimulus. (Intensity corresponds to probability). Top: high-contrast diamond. Bottom: low-contrast diamond At edges, the local likelihood is a ‘fuzzy’ constraint line; at corners, the local likelihood peaks around the veridical velocity. The sharpness of the likelihood decreases with decreasing contrast (because the signal-to-noise ratio gets worse)

Results • Bayesian model maximizes ‘posterior probability’ of velocity estimate given image data P(V|I) = P(I|V)P(V) / P(I) (note: P(I|V) estimate incorporates noise model) • Model exhibits same geometry/contrast interaction as human subjects

Bayesian perceptual model exhibits “optical illusion”: (a) High-contrast thin rhombus estimated to move horizontally (b) Low-contrast thin rhombus estimated to move obliquely

For “fat” rhombus, the model estimates horizontal motion regardless of contrast.

(d) Circles show perceived direction for a human subject as (low contrast) rhombus angle was shifted gradually from thin to fat. Solid line shows the predictions of the Bayesian estimator. Dotted lines indicate the predictions when the free parameter was decreased by a factor of 10 (top dotted line) or increased by a factor of 10 (bottom line).