LEARNING MEMORY Jaan Aru jaan arugmail com Deep

LEARNING & MEMORY Jaan Aru jaan. aru@gmail. com

Deep neural nets

Learning through weight changes

Feedback connections!

But the biggest difference?

The most famous patient: H. M.

Hippocampus Medial temporal lobe Patient H. M.

Types of memory

MEMENTO

Patient H. M. demonstrates that. . . 1) medial temporal lobe is important for memory 2) but it is not necessary for all memory! 3) there is a difference between short-term and long term memory 4) there is a difference between declarative and procedural memory 5) medial temporal lobe is necessary for long term declarative memory

Taxonomy of long-term memory

complementary learning systems (CLS) intelligent agents must possess two learning systems neocortex and hippocampus (MTL) NC gradually acquires structured knowledge HC quickly learns the specifics of individual experiences.

Structured Knowledge Representation System in Neocortex hippocampus

Instance-Based Representation in the Hippocampal System pattern separation: DG is crucial in selecting a distinct neural activity pattern in CA 3 for each experience pattern completion : in CA 3 reactivation of part of the pattern that was activated during storage can reactivate the rest of the pattern

Open question What constitutes an „individual experience“? = when does hippocampus do its dance? Every moment something changes. . New experience whenever a big change in high level semantic features

Replay of Hippocampal Memories

Replay of Hippocampal Memories Circumventing the Statistics of the Environment All the experiences don’t have the same priority! 1. Experience can have different lengths 2. replay is biased towards rewarding events Novel, significant reweighting experiences shapes neocortical learning For the cortex it is the same

One shot learning?

Can we apply it in AI? ? Deep. Mind: ‘experience replay’ interleaved with ongoing game-play biasing replay towards significant events Prioritized experience replay Its success supports the role of the hippocampus in reweighting experiences

external memory

differentiable neural computer

Taxonomy of long-term memory

ENCODING RETRIEVAL CONSOLIDATION

ENCODING But what are these stored features? What gets actually stored?

One image, hundred ways to encode it

How you encode it determines how you remember

ENCODING CONSOLIDATION RETRIEVAL For remembering, try to go mentally back to the situation where you learned it

Encoding what? What you encode determines what you remember! What you encode depends on your mood, attention, other thoughts and associations – it is highly variable Adult humans mostly do not encode sensory features, but rather semantic concepts Transfer learning is hard with sensory features, but easy with semantic concepts

Transfer learning is as good as your features Transfer learning is hard with sensory features, but easy with semantic features As two similar things might be very different in sensory space Whereas semantic features are multisensory / amodal – different sensory features can be combined

Summary ANNs learn structured knowledge Learning single experiences is essential Based on hippocampus Prioritized replay to cortex Applying in AI has been a success (Deep. Mind) One image, hundred ways to encode it Semantic encoding enables transfer