Searchlight analysis neural similarity and psychological similarity 1

Searchlight analysis, neural similarity and psychological similarity 1

Standard f. MRI analysis: what is the number inside each voxel? Raw data: BOLD MRI signal Analysis output: activation in each voxel BUT: What we care about: task-relevant information within each voxel 2

Looking at the patterns in a local spatial neighbourhood: Information-based brain imaging Kriegeskorte et al. , PNAS, 2006 Instead of analyzing one voxel at a time, look at spatial patterns of activity in local neighbourhood • “Sphere of information” • No spatial smoothing: that wipes out local patterns The number in each voxel is not an activation value It is how much information is present 3

Searchlight analysis Pull out a local neighbourhood BOLD image Look at the patterns in that neighbourhood 4

Result: a map of information values Searchlight map made at single-subj level At group level, look for average effects across subjects, just like any other group average Very popular approach 5

Searchlight analysis: Advantages and disadvantages Pros: • Different conditions may yield pattern differences without average-intensity differences • Standard analysis (General Linear Model, GLM) will be blind to such differences, searchlight might see them Cons: • You still just end up with a map as output • A map of where stuff is happening might not tell anything about mechanisms or representations 6

Brain images don’t directly show representations. But they can reveal structure Most neuroimaging is massively univariate • Individual voxel activations just go up and down A multivariate approach • Multivoxel patterns have a similarity structure • Searchlight analysis doesn’t directly look at similarity structure • Another approach (also due to Kriegeskorte!): Representational Similarity Analysis 7

Representations have structure From Tom Griffiths et al. , Trends in Cog Sci, 2010 8

Representations have structure From Tenenbaum et al. , Science, 2011 9

Why care about similarity? Neural point of view: • Simple measures of neural similarity, e. g. correlation between f. MRI patterns, can be surprisingly good at decoding content Behavioural point of view: • Similarity underlies generalisation • “Consequential region” – Roger Shepard Computational point of view: • Similarity is an efficient way of representing the world 10

Shimon Edelman: “Representation is representation of similarities” 11

Representing in terms of similarities can be efficient What is a grapefruit? • A grapefruit is a large fruit of the citrus family, with internal segments arranged a bit like longitude regions in the Earth, growing in warm climates, etc. … • It’s a bit like a big orange, but yellow and sourer 12

Similarity in which sense? Is a gun similar to a banana? • Visual similarity: yes • Functional similarity: no “Similarity” in itself is poorly defined • But in “similarity in a given sense” can be well-defined • Lots of relational concepts are like this, e. g. “to the left” 13

Similarity: Shepard vs. Tversky Roger Shepard: • Similarity can be represented by positions in a space • Similarity is like a distance measure • Symmetric: Similarity(A, B) = Similarity(B, A) Amos Tversky • Similarity is more like set-overlap than like positions in a space • May not be symmetric • Example: How similar is Luxembourg to France? • How similar is France to Luxembourg? 14

Similarity-space The set of pairwise similarities between items, as defined by some similarity-measure (or dissimilarity-measure) Dissimilarities between A, B and C A A 0 B 1 C 5 B 1 0 4 C 5 4 0 A B C 15

Similarity-space: a long history in Cog Psych and Computer Science Roger Shepard (1962), John Kruskal (1964) • Multidimensional scaling (MDS) • Takes a set of similarities, and represents them as the best-fitting lower-dimensional projection Computer vision, visual psychophysics • “Representation is representation of similarities”, Shimon Edelman (1998) Olfaction • Cleland, Sobel, Gottfried Computational analysis of language • Latent semantic analysis (LSA). Landauer et al. • Topic modeling (Griffiths, Steyvers, etc. ) 16

Univariate measures just go up or down, but multivariate patterns have similarity structure Shepard, R. N. Toward a universal law of generalization for psychological science. Science, 1988 Kiani et al, J. Neurophys. , 2007; Kriegeskorte et al. , Neuron, 2008

Neural similarity-spaces can be very informative Kriegeskorte, Kiani et al. , Neuron, 2008 18