Optimal Eye Movement Strategies In Visual Search Visual

Optimal Eye Movement Strategies In Visual Search

Visual Accuity ü http: //www. svi. cps. utexas. edu/Hamilton. Creek. mov

Images from Laura Walker Renninger

http: //www. svi. cps. utexas. edu/foveator. htm

Attention (And Fixation) Shifting Strategies ü Visual Saliency § Attend to what stands out from background ü Experience Guided Search (last class) § Attend to locations of maximum posterior probability (MAP) ü Information Maximization § Given low resolution in parafovea, perhaps we move our eyes to gather as much information as possible. § Low resolution in parafovea -> uncertainty § Move eyes to reduce this uncertainty (= gather information) ü Information maximization and MAP both predict task specificity of eye movements.

Yarbus (1967)

Najemnik & Geisler (2005) ü 1/f noise same statistics as natural images ü Target sine wave grating ü Manipulations Target contrast Background noise contrast

Measuring Visibility At Fovea ü ü ü Subject fixates at center Two displays in quick succession Task: determine which one contains the target grating Measure accuracy as a function of target and background contrast at each location For center location: Threshold = 82% accuracy

Derive Discriminability Curves ü ü For a given background contrast and target contrast d’: signal to noise ratio If noise is Gaussian, 1/d’ 2 is variance of noise distribution Two noise components § external noise: due to 1/f background contrast. 05, target contrast. 07 background contrast. 20 and target contrast. 19 § internal noise: due to inefficiency of sensory system

Terminology d’E(i): discriminability due to external noise at location i d’I(i, k(t)): discriminability at location i due to internal noise given fixation at current time is k(t) Combined noise from two independent Gaussian sources:

Snapshot Likelihood (Observation) Model ü ü Imagine a feature detector at each location that matches the target template (grating) against the visual information at that location Wi, k(t): Observation at location i at time t when fixation at k(t) § Mean = 0. 5 if target present, -0. 5 if target absent § Drawn from Gaussian with variance g[i, k(t)]-1

Integrating Sequence Of Observations ü ü Sequence of t=1…T fixations At each fixation, obtain noisy evidence concerning target presence at each location i § Wk(1), Wk(2), … Wk(T) ü Bayesian ideal observer:

Quiz ü ü Are the W’s independent conditioned on a location (i or j)? It depends on nature of internal and external noise.

Conditioning On External Noise ü ü xi: (unknown) external noise at location I Marginalize out over x: Assuming internal noise is independent over time and space of external noise: And external noise is independent over space:

Conjugate Priors To The Rescue ü Because internal noise and external noise are Gaussian, = +1 if q=i, -1 otherwise ü Integral can be computed analytically. Gaussian prior Gaussian likelihood -> Gaussian posterior

Conjugate Priors To The Rescue ü ü ü Form of likelihood: Form of posterior: But ugly constant is the same in numerator and

Final Result With Related to 1/variance Of observation ü Intuitive result Weighted sum of evidence, where weight ~ reliability Simple incremental rule for computing over time

What We Haven’t Discussed Yet ü How is next fixation location chosen? § Go to location most likely to contain target (MAP location) § Go to location that will obtain the greatest expected reduction in uncertainty (entropy) § Go to location that will obtain information that will maximize probability of correctly identifying target ü Comparison to random searcher § Ideal decision maker, but chooses fixations randomly

Choosing Next Fixation: Some Ugly Details C: correct identification of target Normal density Normal cdf Depends on mean and variance of probability density for an observation

Generating Fixation Sequences

Average Spatial Distribution Of Fixations For 1 st, 3 d, and 5 th Saccades Ideal and Human Observers

Results I ü Median # fixations to locate target, as a function of foveated target’s visibility Background noise contrast =. 025 solid = ideal searcher Background noise contrast =. 20 dashed = random searcher Observer 1 Observer 2

Results II ü Median number of fixations to locate target as a function of target eccentricity (x axis) and target visibility in fovea (d’) Background noise contrast =. 05 Background noise contrast =. 20 Solid = ideal observer Dots = medians (less reliable at small eccentricities)

Results III ü Posterior probability at target location as a function of the number of fixations prior to finding target dashed = random searcher solid = ideal observer

Are Fixations Information Seeking? ü ü Comparison to MAP selection Can’t distinguish

Distribution of Fixations ü MAP selection vs. information seeking

Distribution of Fixations II ü ü Direction of fixations relative to center of display Confirms previous result

Take Home ü Visual search can be cast as optimal § Optimal choice of next fixation § Possibly not optimal integration of information over fixations ü …subject to limitation on quality of visual information § Noise in images § Acuity limitation of retina

Take Home II ü ü We’ve discussed several Bayesian accounts that cast vision and attention in terms of ideal observers. How does this analysis give us insight into how the visual system works? § Rigorous starting point for developing models § Provides well motivated computational framework § Can ask how human behavior deviates from optimal computation § Can ask how people achieve near-optimal performance with imperfect, noisy neural hardware