Modeling Large Scale Neural Systems Barry Horwitz Brain

Modeling Large Scale Neural Systems Barry Horwitz Brain Imaging & Modeling Section National Institute on Deafness & Other Communication Disorders National Institutes of Health 1

Collaborators Malle Tagamets Fatima Husain Theresa Long Brent Warner Julie Fitzer Jieun Kim Allen Braun Yukiko Kikuchi Mort Mishkin Feng Rong Jose Contreras-Vidal 2

Methods to Understand Neural Basis of Human Cognition 1. Brain lesions & cognitive neuropsychology 2. Pharmacological and genetic studies 3. Electrophysiological recordings in primates (mammals) 4. Transcranial magnetic stimulation 5. Functional neuroimaging Hemodynamic-metabolic methods (PET, f. MRI) Electric-magnetic methods (EEG, MEG) All these data are generally incommensurate with one another. 3

Methods to Understand Neural Basis of Human Cognition 1. Brain lesions & cognitive neuropsychology 2. Electrophysiological and lesion studies in primates (mammals) 3. Pharmacological and genetic studies 4. Transcranial magnetic stimulation 5. Functional neuroimaging Hemodynamic-metabolic methods (PET, f. MRI) Electric-magnetic methods (EEG, MEG) Functional neuroimaging is unique – most of the other techniques permit one to examine only one neural entity at a time. Functional brain imaging allows one to study neural networks directly. 4

Problems with Relating Hemodynamic Data to Underlying Neural Activity 1. Spatial resolution - each PET or f. MRI resolvable element contains multiple and diverse neuronal populations. 2. Temporal resolution - temporal resolution of neuronal activity is on the order of milliseconds; PET and f. MRI (because of hemodynamic delay) is on the order of seconds; fast transients may be invisible to PET/f. MRI. 3. Synaptic vs. neuronal activity - electrical activity comes from cell body firings, PET/f. MRI reflect primarily the activity of synapses; excitatory vs. inhibitory. 4. Connectivity - PET/f. MRI activity is a mixture of local and afferent synaptic activity. 5

Large-Scale Neural Modeling Goal: Construct a large-scale, neurobiologically realistic neural model that can perform tasks like those studied by PET and f. MRI. • Multiple, interconnected brain regions (feedforward and feedback connections). • Each region consists of multiple neuronal units (cortical column). • The basic unit consists of an excitatory-inhibitory pair. • Model can perform multiple tasks (e. g. , DMS for shape, control task). • Dynamic behavior of excitatory units in each region matches that observed by primate electrophysiological studies. • Synaptic activity (both excitatory and inhibitory), integrated spatially and temporally, represents r. CBF/BOLD. 6

Uses for Large-Scale Modeling 1. Understand how cognitive and sensorimotor processes are implemented neurally (forward modeling). 2. Test experimental design and data analysis methods. 3. Method to understand the neural substrate for high-level concepts. 4. Method to combine multimodality information (e. g. , f. MRI, MEG, lesion). 7

Perceptual Objects Perceptual Object Subject to figure-ground separation* Examples of visual objects • Nameable object (e. g. , dog, table) • Delimited pattern Examples of auditory objects** • Word • Melodic fragment • Definable & delimited environmental sound *Kubovy and Van Valkenburg, Cognition, 2001 8 ** Griffiths and Warren, Nat. Rev. Neurosci. , 2004

From Signal to Percept to Concept (higher cortical levels) Percept (image) (primary cortex) Roy Patterson et al. Signal (receptors) 9

Neuroanatomy for Visual and Auditory Object Processing Kass et al. 10

Delayed Matched-to-Sample Tasks Frequency Shape Tonal pattern Stimulus 1 Delay Stimulus 2 Response ITI, next trial TIME 11

Cerebral Cortex 8: 310 -320 (1998) 12

Regions of the Visual Model LGN (stimulus) V 1/V 2 V 4 Prefrontal IT 13 (Tagamets & Horwitz, Cerebral Cortex, 1998)

Basic Unit (Cortical column) Basic Unit of Model and Between-Area Connections 60% E 10% 15% I 1. One excitatory (E) and one inhibitory (I) element per unit. 2. Local connections based on anatomical data. 3. Total afferent input ~ 10 -15% local connections. 4. Sigmoidal activation rule. 14

Excitatory E E Horizontal selective units Inhibitory E I FS Horizontal selective units Corner selective units LGN Vertical selective units IT Vertical selective units FR D 2 Attention V 1/V 2 D 1 PFC V 4 15

The Sigmoidal Activation Rule where ini. I (t ) = åw E ki k Ek (t ) + åwki Ik (t ) I k § Multiparameter differential equation § Δ = rate of increase § d = rate of decay 16

Working Memory Module (IT-PF component) PF-s IT (and other areas) excitatory inhibitory s = cue-selective d 1 = delay d 2 = delay+cue r = response e i PF-d 1 PF-r PF-d 2 Modulator of Attention 17

Stimulus and Response 18

Delay Period 19

Response to Match 20

Visual Model 21

Results and Conclusions: Visual Model Simulations with the Full Model (% CHANGE WITHIN AREAS) High Attention to Shape - Low Attention to Degraded Shape V 1/V 2 +3. 1% V 4 +5. 2% Experimental Results +2. 7% IT Prefrontal +2. 5% +3. 5% (Haxby et al. , 1995) +8. 1% +4. 2% +4. 1% Conclusions 1. Electrical activities in each region match exp. results in primates. 2. PET activities in each region match exp. results in humans. 3. Our hypothesis about how different frontal neuronal populations interact is supported. 4. Our hypothesis about relation between integrated synaptic activity and PET/f. MRI data is supported. 5. Hypothesis about role of top-down processing is supported. 22

Neuro. Image 21: 1701 -1720 (2004) 23

Regions of the Auditory Model (Husain et al. , Neuroimage, 21: 1701 -1720, 2004) PFC Ai FS Aii D 1 D 2 FR STG/STS MGN 24

Freq Network Diagram of Auditory Model Up Selective Units Time Up Selective Units FS Freq MGN Time Contour Selective Units Time Down Selective Units Time Ai STG/ STS R D 1 D 2 Freq Time Down Selective Units PFC Time Aii Attention Excitatory E E Inhibitory E I FS = cue selective D 1 = delay selective D 2 = delay+cue selective R = response selective 25

Simulated & Measured Activity in Auditory Regions: Increasing Temporal Windows of Integration Neural Firing Rate Temporal window of increases along the Ai-Aii-STG/STS pathway. ----- 10 Timesteps stimulus Ai Aii ST Time Ai Aii ST Intensity 1. 0 Optical imaging - Chinchilla (Harrison et al. , 2000) Input 0. 5 100 timesteps Timesteps 26

Monkey call-related activity 40 wn 40 0 40 4 40 5 30 30 20 20 10 10 0 40 0 0 500 1000 7 0 -500 40 30 0 500 1000 -500 40 8 0 500 1000 -500 40 9 0 500 1000 -500 40 10 0 500 1000 40 11 0 500 1000 -500 40 12 30 30 30 20 20 10 10 0 0 -500 40 0 500 1000 -500 40 14 0 0 500 1000 -500 40 15 0 500 1000 -500 40 16 0 500 1000 -500 40 17 0 500 1000 18 0 500 1000 19 30 30 30 20 20 10 10 0 0 -500 40 0 500 1000 -500 40 21 0 0 500 1000 -500 40 22 0 500 1000 -500 40 23 0 500 1000 -500 40 24 0 500 1000 30 30 30 20 20 10 10 0 -500 40 0 500 1000 -500 40 28 0 500 1000 -500 40 29 0 500 1000 -500 40 30 0 500 1000 -500 40 31 30 30 30 20 20 10 10 0 0 -500 40 0 500 1000 40 35 0 -500 0 500 1000 40 36 0 -500 0 500 1000 32 0 500 1000 33 20 20 20 10 10 10 500 1000 -500 40 39 0 500 1000 30 30 30 20 20 10 10 0 -500 40 0 500 1000 40 42 0 -500 0 500 1000 40 43 30 30 30 20 20 20 10 10 10 0 -500 0 0 XD 122303 -1 S SPK ch# 4 ID#9 500 1000 -500 0 500 1000 -500 0 0 500 1000 -500 1000 0 500 1000 complex 0 500 1000 41 FM 0 0 500 1000 -500 0 500 1000 44 0 0 -500 34 -500 40 40 30 0 0 1000 27 -500 40 38 0 30 -500 40 37 0 0 20 -500 40 26 30 0 1000 0 -500 40 25 500 13 -500 40 30 0 -500 40 6 0 -500 30 0 PT 40 3 30 -500 others 40 2 30 0 MC 40 1 30 P < 0. 01

Simulation of f. MRI Experiments § f. MRI activity is simulated by spatial and temporal integration of the absolute value of the synaptic activity over 50 msec (which represents the time needed to acquire an f. MRI slice). § This time course is then convolved with a Poisson function representing the hemodynamic delay. § The resulting function is then sampled every Tr sec (volume acquisition time) to yield the simulated f. MRI activity during each scan series. (Horwitz and Tagamets, Human Brain Mapp. , 1999) 28

Experimental Details Timeline of a Trial Stimulus 1 0. 35 sec Delay 1. 0 sec Stimulus 2 0. 35 sec Response 2. 0 sec Next Trial Task: Subjects discriminate between the two sounds they hear based on whether the two sounds are exactly the same. BLOCK design paradigm; task blocks interspersed with rest. • Subjects: 12 normal, right-handed, American English speakers (5 women, 7 men). • • Training: ½ hour before scanning. Subjects performed at 85% correct. Behavioral data collected via button presses. 22 axial slices, collected on a 1. 5 Tesla GE scanner. Analysis conducted using the statistical parametric mapping (SPM 99) software. Preprocessing, fixed-effects analysis 29

Auditory Stimuli vs. Rest Tones R L Tonal Contours 30

Regions of Interest for Auditory Model Ai Aii (red), PFC(green) ST Left z= 9 z=15 z= 3 PFC ST Ai Right Aii z=9 z=3 z=6 31

Model vs. Experiment: Percentage Signal Change % Signal Change (Tonal Contours-Tones) 100. 0 80. 0 Expt_right Model 60. 0 40. 0 20. 0 Ai Aii ST PFC 32

Model vs. Experiment: Percentage Signal Change % Signal Change (Tonal Contours-Tones) 100. 0 80. 0 Expt_right Model 60. 0 40. 0 20. 0 Ai Aii ST PFC 33

Freq Network Diagram of Auditory Model Up Selective Units Time Up Selective Units FS Freq MGN Time Contour Selective Units Time Down Selective Units Time Ai STG/ STS R D 1 D 2 Freq Time Down Selective Units PFC Time Aii Attention Excitatory E E Inhibitory E I FS = cue selective D 1 = delay selective D 2 = delay+cue selective R = response selective 34

Contour-selective neuron 35

General Conclusions 1. We have constructed a large-scale, neurobiologically realistic model of cortical processing of auditory objects. 2. Simulated neuronal activities agree with experimental findings. 3. Simulated f. MRI data agree with experimental findings. 4. Model can account for a number of perceptual grouping observations (human psychophysical performance). 5. Model predicted existence of a type of neuron found in primate cortex. 6. Simulated MEG looks like it will agree with experimental MEG data. 36