Lecture 8 Word Clustering KaiWei Chang CS University

  • Slides: 23
Download presentation
Lecture 8: Word Clustering Kai-Wei Chang CS @ University of Virginia kw@kwchang. net Couse

Lecture 8: Word Clustering Kai-Wei Chang CS @ University of Virginia kw@kwchang. net Couse webpage: http: //kwchang. net/teaching/NLP 16 6501 Natural Language Processing 1

This lecture v Brown Clustering 6501 Natural Language Processing 2

This lecture v Brown Clustering 6501 Natural Language Processing 2

Brown Clustering v 6501 Natural Language Processing 3

Brown Clustering v 6501 Natural Language Processing 3

Motivation example v Cluster 3 a the Cluster 46 dog cat fox rabbit bird

Motivation example v Cluster 3 a the Cluster 46 dog cat fox rabbit bird boy Cluster 64 is was 6501 Natural Language Processing Cluster 64 chasing following biting… 4

Motivation example v Assume every word belongs to a cluster v “a dog is

Motivation example v Assume every word belongs to a cluster v “a dog is chasing a cat” C 3 Cluster 3 a the C 46 C 64 Cluster 46 dog cat fox rabbit bird boy C 8 Cluster 64 is was 6501 Natural Language Processing C 3 C 46 Cluster 8 chasing following biting… 5

Motivation example v Assume every word belongs to a cluster v “a dog is

Motivation example v Assume every word belongs to a cluster v “a dog is chasing a cat” C 3 C 46 C 64 C 8 C 3 C 46 a dog is chasing a cat Cluster 3 a the Cluster 46 dog cat fox rabbit bird boy Cluster 64 is was 6501 Natural Language Processing Cluster 8 chasing following biting… 6

Motivation example v Assume every word belongs to a cluster v “the boy is

Motivation example v Assume every word belongs to a cluster v “the boy is following a rabbit” C 3 C 46 C 64 C 8 C 3 C 46 the boy is following a rabbit Cluster 3 a the Cluster 46 dog cat fox rabbit bird boy Cluster 64 is was 6501 Natural Language Processing Cluster 8 chasing following biting… 7

Motivation example v Assume every word belongs to a cluster v “a fox was

Motivation example v Assume every word belongs to a cluster v “a fox was chasing a bird” C 3 C 46 C 64 C 8 C 3 C 46 a fox was chasing a bird Cluster 3 a the Cluster 46 dog cat fox rabbit bird boy Cluster 64 is was 6501 Natural Language Processing Cluster 8 chasing following biting… 8

Brown Clustering v C 3 C 46 C 64 C 3 C 8 C

Brown Clustering v C 3 C 46 C 64 C 3 C 8 C 46 a dog is chasing a cat P(C(dog)|C(a)) Cluster 46 Cluster 3 a the dog cat fox rabbit bird boy Cluster 64 is was P(cat|C(cat)) Cluster 8 6501 Natural Language Processing chasing following biting… 9

Brown clustering model v C 3 C 46 C 64 C 3 C 8

Brown clustering model v C 3 C 46 C 64 C 3 C 8 C 46 a dog is chasing a cat P(C(dog)|C(a)) Cluster 46 Cluster 3 a the dog cat fox rabbit bird boy Cluster 64 is was P(cat|C(cat)) Cluster 8 6501 Natural Language Processing chasing following biting… 10

Brown clustering model v 6501 Natural Language Processing 11

Brown clustering model v 6501 Natural Language Processing 11

Model parameters v C 3 C 46 C 64 C 8 C 3 C

Model parameters v C 3 C 46 C 64 C 8 C 3 C 46 a dog is chasing a cat Cluster 3 a the Cluster 46 dog cat fox rabbit bird boy Cluster 64 is was 6501 Natural Language Processing Cluster 8 chasing following biting… 12

Model parameters v 6501 Natural Language Processing 13

Model parameters v 6501 Natural Language Processing 13

Log likelihood v 6501 Natural Language Processing 14

Log likelihood v 6501 Natural Language Processing 14

v 6501 Natural Language Processing 15

v 6501 Natural Language Processing 15

v 6501 Natural Language Processing 16

v 6501 Natural Language Processing 16

v 6501 Natural Language Processing 17

v 6501 Natural Language Processing 17

Algorithm 1 v 6501 Natural Language Processing 18

Algorithm 1 v 6501 Natural Language Processing 18

Algorithm 2 v 6501 Natural Language Processing 19

Algorithm 2 v 6501 Natural Language Processing 19

Example clusters (Brown+1992) 6501 Natural Language Processing 20

Example clusters (Brown+1992) 6501 Natural Language Processing 20

Example Hierarchy(Miller+2004) 6501 Natural Language Processing 21

Example Hierarchy(Miller+2004) 6501 Natural Language Processing 21

Quiz 1 v 6501 Natural Language Processing 22

Quiz 1 v 6501 Natural Language Processing 22

6501 Natural Language Processing 23

6501 Natural Language Processing 23

Lecture 2 Ngram KaiWei Chang CS University ofLecture 2 Ngram KaiWei Chang CS University of
Lecture 1 Introduction KaiWei Chang CS University ofLecture 1 Introduction KaiWei Chang CS University of
Group 1 word word word word word wordGroup 1 word word word word word word
Lecture 5 Representation Learning KaiWei Chang CS UCLALecture 5 Representation Learning KaiWei Chang CS UCLA
Lecture 3 MultiClassification KaiWei Chang CS UCLA kwkwchangLecture 3 MultiClassification KaiWei Chang CS UCLA kwkwchang
Lecture 9 Semantic Parsing KaiWei Chang CS UCLALecture 9 Semantic Parsing KaiWei Chang CS UCLA
Lecture 4 Structured Prediction Models KaiWei Chang CSLecture 4 Structured Prediction Models KaiWei Chang CS
KaiWei Chang Joint work with Scott Wentau YihKaiWei Chang Joint work with Scott Wentau Yih
Final Project KaiWei Chang CS UCLA kwkwchang netFinal Project KaiWei Chang CS UCLA kwkwchang net
Lecture 12 Clustering What is Clustering Clustering isLecture 12 Clustering What is Clustering Clustering is
ENSEMBLE CLUSTERING ENSEMBLE CLUSTERING clustering algorithm 1 partitionENSEMBLE CLUSTERING ENSEMBLE CLUSTERING clustering algorithm 1 partition
FLAT CLUSTERING HIERARCHICAL CLUSTERING What is clustering GroupingFLAT CLUSTERING HIERARCHICAL CLUSTERING What is clustering Grouping
DATA MINING CLUSTERING Clustering 4 Clustering unsupervised classificationDATA MINING CLUSTERING Clustering 4 Clustering unsupervised classification
Clustering Kmeans Clustering 2 means Clustering Curve ShowsClustering Kmeans Clustering 2 means Clustering Curve Shows
Kmeans Clustering Kmeans Clustering What is clustering WhyKmeans Clustering Kmeans Clustering What is clustering Why
C 0 clustering Liang Zheng Clustering Clustering alongC 0 clustering Liang Zheng Clustering Clustering along
Variable Clustering 1 Variable Clustering Variable clustering findsVariable Clustering 1 Variable Clustering Variable clustering finds
CLUSTERING Overview Definition of Clustering Existing clustering methodsCLUSTERING Overview Definition of Clustering Existing clustering methods
Clustering Outline Introduction Kmeans clustering Hierarchical clustering COBWEBClustering Outline Introduction Kmeans clustering Hierarchical clustering COBWEB
Stable Clustering Vladyslav Kolbasin Clustering data Clustering isStable Clustering Vladyslav Kolbasin Clustering data Clustering is
Similarity and clustering Clustering Motivation Problem Query wordSimilarity and clustering Clustering Motivation Problem Query word
Similarity and clustering Clustering Motivation Problem Query wordSimilarity and clustering Clustering Motivation Problem Query word