Bootstrapping Tom Griffiths Bootstrapping How to learn words

Bootstrapping Tom Griffiths

Bootstrapping • How to learn words without knowing words • Various proposals: – “semantic bootstrapping” – “syntactic bootstrapping” (Pinker, 1984) (Gleitman, 1990) • Characterized by accelerated learning (e. g. Regier, 2004) • Question: – when is bootstrapping possible?

“blicket” Word learning “blicket”

Bayes’ theorem Posterior probability h: hypothesis d: data Likelihood Prior probability Sum over space of hypotheses

Bayesian word learning (Tenenbaum, 1999; Tenenbaum & Xu, 2002) • Data – scene-word pairs • Hypotheses – functions labeling scenes • Likelihood – weak sampling – strong sampling x h w

“blicket” p(d|h) = 0

“blicket” p(d|h) = 1/3

“blicket” p(d|h) = (1/3)3

“blicket” p(d|h) = 1/12

“blicket” p(d|h) = (1/12)3

Bootstrapping • Bayesian word learning is a form of semantic bootstrapping (Niyogi, 2002) • What about accelerated learning? – non-linear* increase in probability of correct answer for a random scene and word • When can it occur? – not when hypotheses independent and all equally likely, when using weak sampling – speculation: hypotheses are dependent

Forms of dependency • Hierarchical priors – unknowns across learning events • Compositional priors – unknowns within learning events

Hierarchical priors x h h h w “blicket” x x w “toma” x h w “dax” w “wug”

“blicket” “dax” “toma” “wug”?

Hierarchical priors • What is contained in a hierarchical prior? • Any learned information that constrains scene-word mappings – typical referents (whole object) – dimensions of stimuli (shape/substance) – pragmatic dependencies (mutual exclusivity) – sound and meaning (morphology)

Compositional hypotheses “blicket toma” h G x w 1 h 1 w 2 holistic x w 1 h 2 w 2 independent h 1 x w 1 h 2 w 2 compositional

Compositional hypotheses • Good news: – express syntactic bootstrapping – model referential uncertainty • Bad news – requires complete linguistic theory

Bootstrapping • When do we see accelerated learning? – speculation: dependent hypotheses • Sources of dependency in language – hierarchical priors – compositional hypotheses • Bootstrapping goes beyond language – learning causal theories aids learn causal relationships, learning concepts…