Mid level vision neglected yet still important Ken
- Slides: 57
Mid level vision, neglected yet still important Ken Nakayama Harvard University 1
21 st C challenge Existence and variation of occlusion and variable sources of illumination pose unsolved problems for vision 2
• Object representation needs an intermediate level format • Low level vision alone is not even explanatory for wide range of visual processes (motion, stereo, search) • Missing -- a satisfactory scientific description of surface level vision 3
1970 s Visual take-over of the whole brain visual 1980 s 1950 s Half of primate brain and substantial fraction of human brain devoted to vision 4
Macaque monkey brain flattened Visual regions shown in color 5
Global division of the visual system dorsal (where, how) ventral what 6
action BYPASS? object recognition 7
action depth search motion attention object recognition 8
Kanizsa Phenomenology, reviving the Gestalt approach Level: surfaces Method: phenomenology Practitioner: Gaetano Kanizsa new concepts: amodal and modal completion 9
Amodal competion (behind) modal competion (in front) 10
Inferences, but at what level ? 11
12
Suggests that there is a completion process within the visual system Amodal completion trumps knowledge of horses 13
Amodal completion allows fragments to be grouped and thus recognized (strongest evidence) spot the 5 letter Bs letter B same fragments From Bregman, 1990 14
Occlusion and the problem of segmentation for object recognition What belongs together ? Border ownership issues - for 3 -D scenes, borders cannot be shared. Border dispute needs resolution Rule - border belongs to the closest surface 15
Problem of segmentation Kanisza’s figure Normal or amputee ? 16
Border ownership dictated by “lines” prevents modal and amodal completion 17
New sources of evidence Stereoscopic disparity Surface in front “owns” the border. Thus face on right is broken up, on left is OK Nakayama et al. Perception ‘ 89 faces easier to recognize on left 18
Stereoscopic depth also determines border ownership between regions. Nearer surface will own the border (for opaque surfaces) Nakayama & Shimojo stereo demonstrations 19
Image level can’t even explain much lower level vision Deployment of attention, motion perception, texture, visual search 20
image where how dorsal pariet al what vent ral t emporal surf aces 21
Surfaces needed for much lower visual function 22
He and Nakayama search task Used stereo vision Nature (1992) 23
24
Random dot stereogram The correspondence problem: an image based problem unpaired points 25
L. E. only R. E. only 26
invisible to right eye occluding surfaces What gives rise to unpaired points? What would happen if we presented unpaired points by themselves? 27
Da. Vinci stereopsis (Nakayama &Shimojo) 28
Scene depth from unpaired gaps a Gillam and Nakayama, 1999 29
Forest vs plane A plane is a surface Which can occlude, a set of random sticks cannot 30
Planes vs sticks Gillam and Nakayama, 200 31
Level of processing high or low level inference? Hypothesis - inferences learned via associative cortical learning 32
generic view principle when faced with more than one surface interpretation of an image, the visual system assumes it is viewing the scene from a generic, not accidental, vantage point. Nakayama and Shimojo 33
Some counterintuitive observations folded wings? LE RE folded cards? Why don’t we interpolate depth and see folded wings and cards? 34
35
Accidental vs generic vantage points 36
• accidental view • generic view 37
sur f aces images square ( surf ace) cube ( volume) viewing sphere 38
generic view principle when faced with more than one surface interpretation of an image, the visual system assumes it is viewing the scene from a generic, not accidental, vantage point. 39
40
Some counterintuitive observations folded wings? LE RE folded cards? Why don’t we interpolate depth and see folded wings and cards? 41
LE RE conclusion: this is a generic view of crossed bars not wings 42
this is the generic view of transparent surface in front, not a folded card 43
neural mechanisms of surface representation ? Cells in V 2 respond to subjective contours Strategy: vary stimuli in ways that lead to Appearance and disappearance of subjective contours 44
Physiological correlates of illusory contours in single unit recordings Recordings from a single cell in area V 2 of monkey 45
Bakin, Nakayama, and Gilbert, 2000 46
Cells coding Border ownership? Edgar Rubin figure and ground Von der Heydt Et al. Edge labeling? contrast polarity vs edge labeling 47
+ +� -� + +� image based response -� 48
Border ownership based response 49
Border ownership cells Von der Heydt and colleagues 50
Von der Heydt (1984) yes V 2 Bakin, Nakayama, Gilbert (2000) V 2 Da. Vinci stereopsis Border ownership cells (V 2) Mechanistic account of surface representation? --> 51
21 st C challenge Existence and variation of occlusion and variable sources of illumination pose unsolved problems for vision 52
• Object representation needs an intermediate level format • Low level vision alone is not even explanatory for wide range of visual processes (motion, stereo, search) • Missing -- a satisfactory scientific description of surface level vision -- demos the importance for illumination for object recognition 53
importance of shadow processing Ted Adelson 54
Shadow processing in object recognition shadow face yes reduce contrast yes outline no 55
reduce contrast add outline no yes Outline is very destructive to seeing regions as shaded. Line is interpreted as a bounding Contour of an object 56
57
- Mid-level vision
- Still not there yet
- Example of a news story
- From most important to least important in writing
- Least important to most important
- Mid level management
- Human vision vs computer vision
- Hubel and wiesel experiment
- Lamentations 3 22-23 amp
- Lex yacc example
- I done my homework last night
- Trait vs state
- Present continuous tense in hindi
- Pumba she's gonna eat me
- Linda has just walked outside with grandmother
- Draw present perfect tense
- Ever never present perfect
- Just already yet present perfect
- Whats a phrase and clause
- Jeb likes cars but he cant drive yet
- Contrast conjunction
- Compound-complex sentence
- Yarn yet another resource negotiator
- Reasonable yet beyond natural reason
- Grand union flag
- The president yelled we must have peace
- Elaboration transition words
- Literary terms in macbeth
- Personification in tuesdays with morrie
- Don't eat the marshmallow yet
- Fanboys for
- Max isn't home yet. that worries me
- Are we cool yet?
- Yet here's a spot
- They haven't finished their homework yet
- Sonnet 19
- Timeline for acts i-iii
- They are ready question tag
- Semaphores provide a primitive yet powerful and flexible
- Zennioptical com
- Ten things i hate about you poem
- I will show you a still more excellent way
- I am not born yet poem
- Yet herein i will imitate the sun
- Yet another day
- So simple yet so
- Yet not to thine eternal resting place
- If not yet done
- Jeopardylabs present simple and continuous
- Most yet black hole
- Yet in these thoughts myself almost despising
- Oh romeo my romeo
- Tripod sign in polio
- Most yet black hole
- Characteristic features of culture
- Yet once more o ye laurels
- The winner of the race hasn't been announcing yet
- Bash web framework