State Estimation and Prediction Based On Dynamic Spike

State Estimation and Prediction Based On Dynamic Spike Train Decoding Noise, Adaptation, and Multimodality Ron Meir Department of Electrical Engineering Technion, Israel With: Omer Bobrowski and Yonina Eldar

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions The Problem Noise ubiquitous Environment World state Objective: Sensory Processing State estimator Based on partial noisy sensory input estimate world state

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Desiderata Compute posterior distribution P(current state|sensory input) or P(next state|sensory input) Continuous time Online - process each spike upon arrival Real time – fixed computational load Implementation by a recurrent neural network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Summary of Main Results Main assumption: State is a Markov process Mathematical foundations – developed in the 70 s Suggest implementation by a simple bi-linear neural network Extend the original framework: Noisy input History dependent spike trains (exploring adaptation) Multimodal inputs Prediction Demonstrate known results from static-case

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Comparative Dimensions Options Our work Stimulus Static/Dynamic Time Continuous/Discrete Continuous Dynamics General/Markov Space Continuous/Finite Solution Exact/Approximate Exact Neural implementation Yes/No Yes Receptive fields General/specific General Environmental noise Yes/No Yes Adaptation Yes/No Yes Main restrictions – (1) finite-state (2) Markovian

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Problem Setup - A process representing world state - Sensory cell responses Spike trains Partial, noisy, delayed, redundant, … …. . ? Decoding Network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Problem Formulation Main assumptions: World state – finite-state continuous-time Markov process with a generator matrix Sensory activity – Poisson processes with rates - tuning curves Objective: Compute posterior probabilities: Requirements: Online real time computation Neural network implementation

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Mathematical Framework – Nonlinear Filtering History: Formulated in sixties (Kushner, Zakai, Wonham) Extended in seventies to point process observations (Snyder, Segal, Kailath) Our work: Relies on this rigorous theory Demonstrates real-time neural implementation Extensions & Generalizations – multimodality, noise, non-Poisson, …

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Key Concept Zakai Equation: A simple filter for the non-normalized probabilities Meaning: There is a special set of functions Such that: Computing probabilities – hard Computing non-normalized probabilities – easy

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Filtering Equation Stochastic Differential Equation prior sensory data 0 0 no spikes bias

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Neural Implementation Sensory layer Posterior network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Example – Visual Tracking No input Increased uncertainty Object’s trajectory 0 Probability 1

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions System Behavior Between spikes: Spike arrival from the mth sensory cell: Closed form solution available

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions The Static Case Setup: Constant input, selected from finite set Assumptions: Gaussian* prior: Gaussian tuning-curves: Posterior: Gaussian* flat prior Population vector (Georgopoulos 82)

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Generalization and Extensions

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Environmental Noise Previous setup: Sensory system hidden state Posterior network non-normalized probabilities Limitation: Sensory response is a direct function of the state Noisy setup: Noisy channel Sensory system hidden state Rates are functions of the noisy state Posterior network non-normalized probabilities

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Environmental Noise Assumptions: - a finite-state Markov process ( ) are independent Tuning-curves: • E. g. , additive noise - Solution: Key observation: is a Markov process ( Write recursive equation computing Compute the marginal non-normalized probabilities )

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Environmental Noise Result: where Average sensory response (with regard to the noise)

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Environmental Noise

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Optimal Tuning Curve Width Motivation: C High precision D Poor coverage D Few spikes Expected: optimal tuning curve width D Low precision C Good coverage C Many spikes

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Optimal Tuning Curve Width Setup: Static stimulus Additive noise: Assumptions: Gaussian* prior Gaussian* noise Gaussian tuning-curves Optimality criterion: MSE of the optimal estimator -

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions History Dependent Spike Trains Why not Poisson? Lack memory Physiological phenomena Refractory period – firing is exhausting Adaptation – neurons get bored

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions History Dependent Spike Trains Self-exciting point processes: Rate depends on history – Includes – Poisson, Renewals, and more… Equation: (Synaptic depression ? )

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions History Dependent Spike Trains Using spikes efficiently

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions History Dependent Spike Trains Self inhibition term: No adaptation: With adaptation: cells near the true state are the least inhibited

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration Setup: Two or more input modalities (e. g. visual and auditory) Goal: Compute posterior based on both modalities quack A V

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration Reminder: unimodal equation Multimodal equation where multi-sensory data - number of visual/auditory sensory cells - mth visual/auditory input activity - mth visual/auditory cell’s tuning-curve

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Neural Implementation Sensory layer Posterior network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration Possible question: Is it the same as taking inputs of the same modality? Answer: no Biologically • Different information conveyed by different tuning-curve properties (shape, latency, …) Mathematically • Different noise processes • Benefit - two noisy observations instead of one

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration Example: Auditory input compensates for the lack of visual input And vice versa

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration Comparing multimodal vs. unimodal computation - constant - varying Multimodal – inputs of one modality and of another Unimodal – inputs of the same modality

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration – The static case Setup: Constant input, multimodal observations Well studied case: Deneve, Latham, Pouget, others Assumption: Gaussian unimodal posteriors image visual posterior network sound Result: Gaussian multimodal posterior image sound multimodal posterior network auditory posterior network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Multisensory Integration – The static case In our framework: Assumptions: Gaussians tuning-curves and prior (as in the unimodal case) Result: Gaussian multimodal posterior, with: flat prior

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Prediction Objective: Compute posterior distribution of future states Solution: Define: Then: (non-normalized probabilities of future state) (when transition-matrix is regular) Substitute into the original equation to get: where

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Summary Spike based filtering of a continuous-time Markov process Online implementation using bi-linear neural networks Mathematically rigorous treatment of continuous time No temporal information lost Multimodal effects easily accounted for Many extensions noise, history-dependent spike trains, prediction, and more… Demonstrating static-case known results

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Extensions Learning and adaptation – Distributed and robust representation Continuous state-space Physiological interpretation and implementation Biological experiments

!Thanks

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Prediction Sensory layer Posterior network Prediction layer

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Pitkow, Sompolinsky & Meister (PLo. S Biology, 2007) Setup: Static stimulus – horizontal/vertical bar Dynamic noise – fixational eye movements (2 D random walk, continuous time, discrete space) Sensory activity – independent Poisson spike trains Firing rate represents probability Purpose: Determine object’s retinal position Distinguish between horizontal/vertical bars Computation by a rate-based recurrent neural network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Pitkow, Sompolinsky & Meister (PLo. S Biology, 2007) Dynamic posterior update: stimulus S at position x, given spiking activity up to time t spike train generated by retinal neuron y tuning-curve related functions discrete-space differential

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Pitkow, Sompolinsky & Meister (PLo. S Biology, 2007) Pitkow et al. (PLo. S Biology, 2007)

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Pitkow, Sompolinsky & Meister (PLo. S Biology, 2007) Summary: Equation & neural implementation similar to our work Provides evidence for biological plausibility in V 1 Compares to psychophysical experiments Comparing to this work: Handles 2 D random walks only Does not go beyond the basic equation (e. g. noise, prediction, multisensory, adaptation, etc. )

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Deneve (Neural Computation, 2008) Setup: Dynamic stimulus – a continuous-time binary Markov process Sensory activity – Poisson spike trains Neurons encode probability as an inner state Purpose: Compute the log-likelihood ratio Represent the ratio in a single cell activity

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Deneve (Neural Computation, 2008) Summary: A differential equation for computing the log-ratio A single cell mechanism to propagate probability Providing learning mechanisms Comparing to this work: Binary stimulus only Nonlinear equation Inner state (posterior distribution) computation mechanism is not studied Neural implementation might lose information The log-ratio equation is a special case of framework suggested in our work

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Beck & Pouget (Neural Computation, 2007) Setup: Dynamic stimulus – a continuous-time Markov chain Sensory activity – abstract Firing rate represents probability Purpose: Determine the stimulus state from sensory input Computation by a rate-based recurrent neural network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Beck & Pouget (Neural Computation, 2007) posterior state distribution, given spiking activity up to time t stimulus’ generator matrix unspecified functions of the input spikes

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Beck & Pouget (Neural Computation, 2007) Summary: Posterior calculation by a recurrent neural network Comparing to this work: Approximated derivation Quadratic equation Unspecified input terms

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Rao (Neural Computation 2004, NIPS 2005) Setup: Dynamic stimulus – a discrete-time Markov chain Sensory activity – abstract Firing rate represents probability Purpose: Determine the stimulus state from sensory input Computation by a rate-based recurrent neural network

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Rao (Neural Computation 2004, NIPS 2005) Summary: Posterior calculation by a discrete-time recurrent neural network General structure similar to previous models Comparing to this work: Discrete time Approximated derivations (log of sum, random spikes) Nonlinear interactions Unspecified input terms

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Brown, Barbieri, Solo & Co. Setup: Dynamic stimulus – a discrete-time Gaussian AR models. Sensory activity – independent Poisson spike trains Assumed receptive fields – Gaussian/Zernekie polynomials Purpose: Computerized decoding of information from neural activity (Kalman based approach) Highly different purpose than our work implies different requirements/assumptions/approximations

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Recent Work Overview Brown, Barbieri, Solo & Co. Main differences with our work: Implementation: computer vs. neural network discrete vs. continuous time Nonlinear equations, required iterative solutions vs. simple bi-linear equations Model assumptions: discrete-time, continuous-space (Gaussian AR) vs. continuous-time, discrete-space (finite-state Markov) special vs. generic receptive fields Gaussian approximation for the posterior vs. no approximation

Real Time Spike Train Decoding By Neural Networks Overview èProblem èSolution èExtensions èConclusions Example – Multi-target Tracking Combined state-space: Marginal probabilities: