Visual Perception in Humans and Machines Kostas Daniilidis

Visual Perception in Humans and Machines Kostas Daniilidis Assistant Professor GRASP Lab University of Pennsylvania 1

Examples • How do we (humans) recognize faces ? Make a machine find President Clinton’s face in the web 2

3

An interdisciplinary definition • Computer Vision is devoted to the discovery of algorithms, representations, and architectures that embody the principles of visual capabilities. • What are visual capabilities? – Recognizing objects and faces – Estimating shapes and distances – Moving, grasping 4

Relation to other fields • Computer Vision is inspired from Biological Vision (Phenomenology and Models in Psychophysics and in Neurobiology) but does not try to imitate the nature's architecture or algorithms. • Biological Vision and Psychophysics may find computational models discovered in Computer Vision useful for explaining nature. 5

Target problem in computer vision • Compute properties of the 3 D world from one or more digital images • These properties may be – dynamic (observer and object motion) – geometric (distances, object shape) – enabling recognition • The result may be an action (grasp an object, avoid an obstacle) 6

What is an image ? • A gray-value image is just a set of numbers (usually from 0 to 255) 7

An image is a set of numbers 175 189 190 188 199 197 196 193 181 189 191 194 198 196 191 179 189 191 197 198 200 195 173 129 192 194 198 200 194 161 116 198 200 190 152 113 116 119 201 202 185 135 103 114 119 205 180 121 89 104 101 109 114 177 105 88 90 103 101 105 8

An image is a surface I(x, y) 9

Basic image processing operations • Smoothing and Noise Removal 10

Blur removal 11

Edge detection • X derivative • Gradient magnitude • Y derivative 12 • After thresholding

(Sub) sampling • Shannon Theorem: Sampling frequency must be greater than the maximal frequency in the image (therefore smooth before subsample) 13

Brightness perception 14

How do we perceive distances? • Perspective distortion in texture, contour, shading, and a-priori knowledge • Stereopsis (what most people believe) • Motion 15

The Eye as a Pinhole Camera: Perspective Projection Z X u = X/Z 16

Ames’ Illusion 17

Perspective Illusions • A-priori-knowledge bias 18

Quiz • From which points in space is a rectangle viewed as a square (more difficult: an ellipse viewed as a circle ? • Be careful: The center of the ellipse in the image is not the projection of the center of the ellipse on the floor! 19

The power of vanishing points • Perspective projection preserves cross-ratio = AC/AD : BC/BD is the same on the street and in the image. If A is a vanishing point AC/AD = 1. We measure A, B, C, D in pixels in the image A and form cross ratio for image and for the street. BC is computed from the equality of the two ratios. B C D 20

Stereopsis Infer depth from the disparity between the positions of the same feature in left and right image 21

Stereo Reconstruction 22

Stereo-disparity estimation • Search at every pixel for candidates of maximum correlation between left and right • Estimate 3 D-coordinates of point 23

The power of visual motion • Most of the animals have monocular vision (left and right visual fields do not overlap) • 8% of the population can not see stereo • Stereopsis is limited to a very short depth of field (10 m). 24

25

Kinetic depth effect (moving dots) 26

Self- and object-motion 27

Motion artifacts 28

Structure from Motion Given a sequence of images find 1. Ego-motion 2. 3 D-structure 3. Independent motions applying only the assumption of rigidity. 29

Motion Field and Heading Direction 30

Depth map 31

Temporal aliasing Wagon Wheel Illusion: A wheel with a periodic radial pattern is perceived to move backwards depending on the relation between the speed, the radius of the wheel, and the period of the pattern (www. cstr. ed. ac. uk/~rjc/wagon. Wheel) 32

Aperture problem Inside a small aperture displaying a small line we can estimate only the motion direction perpendicular to the line. 33

• My wife and my mother-in-law The role of the focus of attention 34