CBOW Continuous Bag Of Words Model 2017 9
CBOW : Continuous Bag Of Words Model 2017. 9. 20 숭실대학교
Represent the meaning of word – word 2 vec 2 basic neural network models: • Skip-gram (SG) : predicting surrounding context words given a center word • Continuous Bag of Word (CBOW) : predicting a center word from the surrounding context words. CBOW Skip-gram 2 / 17
Continuous Bag of Words model (CBOW) The cat jumped over puddle CBOW 3 / 17
Continuous Bag of Words model (CBOW) |v| Black box 4 / 17
CBOW Example. “ The cat jumped over puddle “ 5 / 17
Step 1: generate one-hot vector the 1 cat jumped center word over puddle 0 0 0 1 0 0 0 1 0 0 6 / 17
Step 2: get embedded word vector … hidden layer 7 / 17
Step 2: get embedded word vector 8 / 17
Step 2: get embedded word vector 9 / 17
Step 2: get embedded word vector 10 / 17
Step 3: average vector average 11 / 17
Step 4: generate score vector … 12 / 17
Step 5: softmax Function 1 0 0 3 2 1 5 2 13 / 17
Continuous Bags of Word Model 14 / 17
Cross entropy -log(x) 그래프 15 / 17
Cross Entropy Cross entropy example -log(x) 그래프 16 / 17
Cross entropy 17 / 17
- Slides: 17