Episodic Recognition Memory for HighDimensional Human Synthetic Faces

Episodic Recognition Memory for High-Dimensional Human Synthetic Faces Yuko Yotsumoto, Hugh R. Wilson, Michael J. Kahana, Robert Sekuler Brandeis University, York University M 076 Purposes 1. To examine how series of briefly-presented, synthetic human faces are represented in memory. 2. To determine what information subjects use when they make episodic recognition judgments. Stimuli were Synthetic Human Faces Synthetic faces were derived from 37 measurements on each original photograph. Images derived from the original faces were filtered and normalized. A tale of two models (*2) We have worked with two models of episodic visual recognition memory. Each assumes that stimuli are encoded and stored separately in memory as separate, noisy exemplars. Stimulus Oriented Recognition Model (STORM) assumes that judgments are based solely on the similarity between the probe item and the most similar stored representation. The probe is judged as a target when the similarity between the probe item and the most similar study item is less than the criterion (Zhou et al, in press). METHODS Simulation with STORM (*4) The percentage of YES response was predicted by STORM for 30 sets of stimuli. All stimulus sets comprised three study items. Table: r 2 obtained by model predictions Prediction by perceptual similarity Prediction by physical similarity Subject 1 0. 5098 0. 3450 Subject 2 0. 6833 0. 3281 Subject 3 0. 4859 0. 2845 Model predictions were compared with data from three human subjects. Noisy Exemplar Model (NEMO) assumes the judgments are based on the summed similarity between the probe item and each stored representation; the summed similarity computation takes account also of the similarity of one study item to another. The probe is judged as a target when the summed similarity exceeds the criterion (Kahana et al. , 2002). *1 One, two, three, or four study items were presented sequentially. Duration for each face was 110 msec, and ISI was 200 msec. Then, a warning tone was followed by a Probe face which was presented for 250 msec. The probe delay was 1200 msec. 8 subjects participated in this experiment. The subjects judged whether the Probe face had or had not been among the Study items. Feedback was provided by distinctive tones. As is customary in Sternberg paradigm, on half the trials, the Probe item had been in the Study set, and on half the trials it had not. 8 subjects participated in this experiment. *2 Given a list of items, , and a probe item, p, NEMO will respond “yes” if Where is the similarity between two representations, p and , is a scaling parameter applied to the similarity between p and each of the study items. is a vector representing the noise associated with each stimulus dimension, and represents the optimal criterion for a list of L items. Given a list of items, S 1, SL, and a probe item, p, STORM will respond “yes” if We used synthetic faces based on three different females, A, B, and C. The vectors representing A, B, and C were tranformed to be mathematically orthogonal from each other. Face D was diagonal of these three orthogonal vectors. A mean female face was made by averaging the measurement vectors for 40 different faces. The stimulus faces were gradually deformed from the mean face to Face A, B, C, or D. A B S 1 S 2 S 3 How are the synthetic faces represented perceptually? C MEAN Perceptual distances among synthetic faces from multidimensional scaling D 4% Each point represent one synthetic face. Distances between points represent pairwise perceptual distances (dissimilarity) between faces. 8% 12% 16% 20% Sternberg Memory experiment with Synthetic Faces (*1) Did you see the Probe Face (P) in the Study set (S 1, S 2, S 3) ? Sound Perceptual representations of synthetic faces (*3) P Duration: 110 ms ISI: 200 ms Delay before probe: 1200 ms List length: 2, 3 or 4 study items 8 subjects, 1536 trials each Factors that might explain discrepancies between the data and predictions by STORM: - did not take account of individual differences in perceptual representation - STORM considers only the single study item most similar to probe; ignores all other study items. - STORM ignores study items’ similarity to one another Several different forms of similarity affect visual memory (*5) Stress =. 25879, RSQ = 0. 62321 Individual differences among subjects Applied ANCOVA to a new set of recognition data in order to evaluate dependence of “yes” responses on various forms of similarity among stimuli. Each point represent one subject. The distribution of points represent the individual differences of perceptual representations of synthetic faces. Angular variation = 0. 0062, 1) Perceptual similarity between probe and the most similar study item (STORM’s only computation) Standard deviation = 0. 1113 Similarity increased the proportion of “yes” response (p<. 01). How different is the perceptual domain from the physical domain? Where is the smallest similarity between p and , is a single scaling parameter applied to the similarity between p and the study item. is a vector representing the noise associated with the stimulus dimension, and C represents the optimal criterion for a list of L items. *3 The similarity matrix was generated by Triadic Comparison. Three faces were presented simultaneously for 500 ms, and subjects chose the one most different from the others. To control the number of trials, balanced incomplete block design was used, and each triad was tested for 30 times (Weller & Romney, 1988). 2100 triadic comparisons were obtained for each of 15 subjects. Perceptual similarity among faces was determined by Multidimensional Scaling, which included models for individual difference. *4 The percentage of YES response was predicted by STORM for 30 sets of stimuli. All stimulus sets had 3 study items. The model responded “yes” if where is the smallest perceptual similarity between probe and study items derived from the result of Multidimensional Scaling, is a vector representing the noise associated with each stimulus dimension, C represents the optimal criterion for a list of three items, and is a single scaling parameter applied to the similarity between p and the most similar study item. The ratio of standard deviation of the noise distribution was fixed to S 1: S 2: S 3=2: 2: 1. *5 2) Summed perceptual similarity between probe and all study items Correlation between distances among faces in the physical domain and in the perceptual domain. Similarity increased the proportion of “yes” responses (p<. 01). =0. 2920 This experiment was done as part of course work at Brandeis University. Twenty nine Brandeis Undergraduates participated in a one hour experiment; each subject gave 436 trials. All were naïve to the purpose of the experiment. The experiment was the same as the first memory experiment with Sternberg paradigm except for the following: There were always 3 study items. Study items and probe were presented for 250 msec each. The ISI was 200 msec, the pre-probe delay was 1200 msec. Result 1: Effect of List Length Recognition declined as number of study items increased from 1 to 4. 3) Perceptual similarity among the study items themselves Regression analysis with hierarchical multiple regression model Similarity decreased the proportion of “yes” responses (p=. 035). Dependent variable : perceptual distance from the mean face Predictor 1 : percentage of the deformation (4, 8, 12, 16, or 20%) Predictor 2 : face type (A, B, C, or D) Result 2: Effect of Probe Position Most recently seen faces were recognized better. Memory experiment with synthetic faces showed the major characteristics found in experiments with lower dimensional stimuli, such as compound gratings. Percentage of deformation from the mean face significantly affects location in MDS similarity space (p<. 01). Face type, A, B, C or D, also significantly affects the perceptual location in similarity space (p<. 05). Perceptual representation of synthetic faces is related to, but not identical to their physical characteristics. Conclusions Visual recognition with synthetic face stimuli resembled previous work with lower dimensional stimuli (compound sinusoids). The representation of synthetic faces were measured and used to examine the validity of visual memory model. The simulation by STORM predicted some amounts of variation of the performance, but a certain amount of variation remained to be unexplained. Multidimensional scaling for individual subjects and the simulation with NEMO would probably increase the accuracy of predictions. References Kahana M. J. & Sekuler R. (2002) Recognizing spatial patterns: A noisy exemplar approach. Vision Research, 42, 2177 -2192. Sternberg, S. (1966). High speed scanning in human memory. Science, 153, 652 -654. Weller SC. & Romney AK. (1987) Systematic Data Collection: Qualitative research methods, volume 10. SAGE Publications Wilson H. R. , Loffler G. & Wilkinson F. (2002) Synthetic faces, face cubes, and the geometry of face space. Vision Research, 42, 2909 -23. Zhou F. , Kahana MJ. & Sekuler R. (in press) Short-term episodic memory for visual textures: A roving probe gathers some memory. Psychological Science.