Biological and cognitive plausibility in connectionist networks for

Biological and cognitive plausibility in connectionist networks for language modelling Maja Anđel Department for German Studies University of Zagreb

Connectionist networks • AI networks for simulating cognitive processes – language • implementing the “computer metaphor” • architectural inspiration: human brain (network of artificial neurons)

Connectionist networks • subsymbolic, not symbolic processing = no division in hardware/software – software “built in” to the hardware structure

Connectionist networks

Language modeling in connectionism Biology / neurological functioning Computer implementation Language processes (as cognitive processes)

Models in history (’ 80) • Rumelhart & Mc. Clelland (1986): English past tense – modeling the Ushaped learning curve

Models in history (’ 80) • Mc. Clelland & Kawamoto (1986): w i c k e l f e a t u r e s Thematic roles in sentences w i c • “How to represent i c k sequences? ” c k e l e l f e. . .

Models in history (’ 80) • Elman (1990): Words of different length • “How to represent sequences? ” outputs hidden units inputs context units

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – – biological realism distributed representations inhibitory competition bidirectional propagation of activation (top-down and bottom-up) – error-driven task learning – Hebbian learning New algorithms that satisfy all the constraints!

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – biological realism

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – distributed representations

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – inhibitory competition

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – bidirectional propagation of activation (top-down and bottom-up) conceptual perceptual dog

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – error-driven task learning desired outcome error computation transfer function error backpropagation input signal

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – error-driven task learning network state transfer function input signal desired outcome error computation

Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: – Hebbian learning "The general idea is an old one, that any two cells or systems of cells that are repeatedly active at the same time will tend to become 'associated', so that activity in one facilitates activity in the other. " (Hebb 1949, p. 70) "When one cell repeatedly assists in firing another, the axon of the first cell develops synaptic knobs (or enlarges them if they already exist) in contact with the soma of the second cell. " (Hebb 1949, p. 63)

Today – biological plausibility • New algorithms – combination of new principles • Complexity brings better results • Old models sucessfully transposed – morphology acquisition – syntax processing – semantic categorization • Computational neuroscience

Thank you!