Multiple Memory Systems The neuroscience of conscious memory

Multiple Memory Systems: The neuroscience of conscious memory Kevin Schmidt AFRL 711 th HPW/RHBNC Northwestern University

Patient H. M. • Medial temporal lobe removed due to epilepsy • Could NOT learn new faces, scenes, words (Scoville and Milner 1957)

Patient H. M. • He COULD learn new skills (e. g. , mirror drawing; Milner 1962) • He learned rapidly and efficiently • BUT on each test day had no memory of having ever practiced the task before. • These findings show that memory is not one entity

Squire and Dede, 2015

Conscious memory • The kind of memory we typically have in mind when we use the term memory in everyday language • Available as conscious recollection 1. Semantic memory • Facts about the world 2. Episodic memory • The ability to re-experience a time and place specific event in its original context

Non-conscious memory • Procedural knowledge, implicit, nondeclarative • What is learned is embedded in acquired procedures and is expressed through performance • More accurate to say that individuals have acquired a disposition to perform in a particular way than to say they have acquired a fact about the world. • Skill-based information • e. g. , Habit memory • Neostriatum (not the medial temporal lobe) • Important for gradual feedback learning (Knowlton, Mangels, Squire, 1996)

Behavioral Data • Serial Interception Sequence Learning (SISL) • A repeating 12 -cue sequence is embedded within scrolling cues • The task speeds up adaptively to keep overall accuracy at ~80% correct • Participants have no explicit knowledge of this sequence, but are more accurate on the sequence than foils • SISL explicit Pre-Training does not impact performance Reber, 2013

Principles of Implicit Learning • Gradually learning across experience to extract statistical co-occurrences among environmental stimuli (or features) • “The Brain’s machine learning algorithm” Reber, 2013

Operational Principles • The memory systems of the mammalian brain operate independently and in parallel to support behavior Nomura and Reber, 2012

Dual-process Artificial Intelligence • System 1 • Reinforcement-based mechanisms • Value of stimuli and actions are learned incrementally and through repeated experience • Extracts statistical co-occurrences among features • System 2 • Episodic memory • Instance based mechanisms • Allow experiences to be encoded rapidly (in ‘‘one shot’’)

Memory Consolidation • Complementary Learning Systems (Mc. Clelland, Mc. Naughton, O’Reilly, 1995) • Memories first stored in the hippocampus • Hippocampus supports reinstatement of recent memories in the neocortex • Neocortical synapses change a little on each reinstatement • Interleaved learning and catastrophic interference

Replay Buzsaki

Hippocampal-dependent memory consolidation

Targeted Memory Reactivation Rudoy, Voss, Westerberg, Paller, 2009

Neuro-Inspired AI, Deep Q-Net (Mnih, 2015) • Expert play on Atari 2600 video games • Transform a vector of image pixels into a policy for selecting actions (e. g. , joystick movements) • Directly inspired by how the multiple memory systems in the brain might interact. • The replay buffer in DQN might = hippocampus • Replay • Network stores a subset of training data in an instance-based way, and then “replays” it offline, learning anew from successes and failures that occurred in the past • Supports interleaved training of deep neural network or neocortex • Avoids the destabilizing effects of learning from consecutive correlated experiences

Deep Q-Net Replay (Mnih, 2015)

Progress • What kind of experiments offer progress in both related fields of replay? • e. g. , direction/control of information flow • Simulating forms of memory loss by disconnecting pieces of the algorithm? • Preplay? • Generative networks (e. g. , GANs)?

Preplay Buzsaki