Computation of pattern invariance in brain like structures

Computation of pattern invariance in brain -like structures (S. Ullman, S. Soloviev) 9. 012 Presentation by Alex Rakhlin March 16, 2001

The problem of shift invariance Visual system recognizes familiar objects despite changes in retinal position. Simple transformation, yet no satisfactory and biologically plausible models.

Main approaches Different initial representations presumably reach a common unified representation at some high levels. Two approaches: Full Replication n Specialized neuronal mechanism dedicated to the detection of a given shape at a given position. n Highly redundant. n Some variations with detection of only simple features. Problem: spatial relation is lost.

Main approaches Normalized Representation n Transform image into normalized central representation, common to all retinal positions. Then let the pattern analyzing mechanisms operate on this common representation. n Requires a complex, unrealistic network. n Does not generalize to other invariances such as rotations. n Implies shift invariance for arbitrary novel shapes. n Does not account for the main properties of units along the visual pathway; does not account for the role of learning. Other models lie in between these two models.

Psychophysical studies High degree of position invariance for line drawings of familiar objects. (Biederman and Copper, 1991) Increase in recognition latencies with discrepancy in size between learned and viewed shapes (Bricolo and Bülthoff, 1993) Significant decrease in discrimination of novel patterns at nearby locations! (Nazir and O’Regan, 1990) Extensive training to discriminate between similar novel patterns at one location doesn’t improve performance at a new location! (Dill and Fahle, 1997, etc) Shift invariance is not automatic and universal.

Shift invariance by the conjunction of fragments Uses full replication at the level of object fragments. Stores view-fragments of different complexity as well as the equivalence relations among them.

Shift invariance by conjunction of fragments

Problem of spatial relations … solved by the use of multiple, overlapping fragments.

First domain: line drawings • Grid = nxn • Shapes = connected figures • Parts = connected pairs of line segments

First domain: line drawings Use small number of parts. Number of shapes grows exponentially with n 2. Number of possible parts grows polynomially with n. In the case of 3 x 3, used several hundred parts for tens of thousands of input shapes. Tested whether the same collection of parts could generate a different shape.

Second domain: image patches Transform gray-level images into binary (edgedetecting filter) Start with small micro-patterns of 3 x 3 patches (512). Next, use 3 x 3 patches to construct larger micropatterns of size 5 x 5 (contains nine 3 x 3 patches). Question: are all 5 x 5 patterns unambiguously determined in terms of the sub-units?

Ambiguity of conjunction 1 2 267549318 3 268547319 4 5 6 7 8 9 269548317

Building the hierarchy Construct shift-invariant units for the larger patterns by a convergence of the more elementary sub-units within a region. If hierarchy is not used, representation becomes increasingly ambiguous as the size of the pattern increases (i. e. constructing 7 x 7 patterns from 3 x 3 directly)

Building the hierarchy In the hierarchical construction the ambiguity is significantly reduced. Ambiguous patterns are visually quite similar! Degree of overlap is a natural measure. System will eventually contain units encoding fragments of different size and complexity. Then these local fragment detectors converge to a global unit, which responds to presence of the fragment anywhere in the region.

Building the hierarchy Novel pattern will activate at two locations a similar set of micro-patterns. Thus, system will be able to immediately generalize. Scheme doesn’t rely on the detailed shape of the object’s bounding contour – tolerant to clutter and occlusion (why? ) Learning: if shape cannot be represented with existing patterns, system will eventually store additional fragments. Need to store this at each point, as objects move in the world. This agrees with psychophysics.

Natural patterns: high orientation preference

Parallels with visual system Increase in receptive field size along the pathway; selectivity to increasingly complex shapes. High degree of parallelism. Memory-based, using sub-patterns seen in the past. Simple computations (rather than complex internal shifting) Fast computation (rather than lengthy iterative)

Extensions 3 D Classification n Equivalence defined between image fragments on the basis of a substitution relation

Conclusions Outlined an approach to the computation of pattern invariance. Learning-based. Image fragments are building blocks for increasingly complex representations. Many similarities with visual system. Agrees the psychophysical studies.

Critique Rotation invariance? Depth invariance? Not clear how to implement the learning mechanism on the level of neurons (i. e. how the convergent representation is created)