Lecture 10 learning word embeddings Glove Debiasing word

Lecture 10: learning word embeddings Glove Debiasing word embeddings Alireza Akhavan Pour CLASS. VISION SRTTU – A. Akhavan Lecture 10 1 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Glo. Ve word vectors SRTTU – A. Akhavan Lecture 10 2 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Glo. Ve(global vectors for word representation) I want a glass of orange juice to go along with my cereal. C, t q. Glo. Ve is another algorithm for learning the word embedding. It's the simplest of them. q. Xij = # times i appears in context of j q. Xij = Xji if we choose a window pair, but they will not equal if we choose the previous words for example. In Glo. Ve they use a window which means they are equal [Pennington et. al. , 2014. Glo. Ve: Global vectors for word representation] SRTTU – A. Akhavan Lecture 10 3 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Glo. Ve Model t c f(x) - the weighting term, used for many reasons which include: q The log(0) problem, which might occur if there are no pairs for the given target and context values. q Giving not too much weight for stop words like "is", "the", and "this" which occur many times. q Giving not too little weight for infrequent words. Theta and e are symmetric which helps getting the final word embedding. SRTTU – A. Akhavan Lecture 10 4 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

royal A note on the featurization view of word embeddings gender you can't guarantee that the axis used to represent the features will be well-aligned with what might be easily humanly interpretable axis like gender, royal, age. SRTTU – A. Akhavan Lecture 10 5 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Debiasing word embeddings SRTTU – A. Akhavan Lecture 10 6 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

The problem of bias in word embeddings q. Man: Woman as King: Queen q. Man: Computer_programmer as Woman : Homemaker q. Father : Doctor as Mother : Nurse Word embeddings can reflect gender, ethnicity, age, sexual orientation, and other biases of the text used to train the model. [Bolukbasi et. al. , 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings] SRTTU – A. Akhavan Lecture 10 7 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Addressing bias in word embeddings Non-bias (299 D) doctor 1. Identify bias direction. babysitter grandmother girl grandfather boy he bias (1 D) q Calculate the difference between: o ehe - eshe o emale - efemale o. . Average 2. Neutralize: For every word that is not definitional, project to get rid of bias. she [Bolukbasi et. al. , 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings] SRTTU – A. Akhavan Lecture 10 8 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Addressing bias in word embeddings Non-bias (299 D) doctor 1. Identify bias direction. babysitter grandmother girl grandfather boy he bias (1 D) q Calculate the difference between: o ehe - eshe o emale - efemale o. . Average 2. Neutralize: For every word that is not definitional, project to get rid of bias. she [Bolukbasi et. al. , 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings] SRTTU – A. Akhavan Lecture 10 9 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Addressing bias in word embeddings Non-bias (299 D) doctor 1. Identify bias direction. babysitter grandmother girl grandfather boy he bias (1 D) q Calculate the difference between: o ehe - eshe o emale - efemale o. . Average 2. Neutralize: For every word that is not definitional, project to get rid of bias. she [Bolukbasi et. al. , 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings] SRTTU – A. Akhavan Lecture 10 10 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

Addressing bias in word embeddings Non-bias (299 D) doctor 1. Identify bias direction. babysitter grandmother girl grandfather boy he she bias (1 D) q Calculate the difference between: o ehe - eshe o emale - efemale o. . Average 2. Neutralize: For every word that is not definitional, project to get rid of bias. 3. Equalize pairs. [Bolukbasi et. al. , 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings] SRTTU – A. Akhavan Lecture 10 11 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ

ﻣﻨﺎﺑﻊ • https: //www. coursera. org/specializations/deep -learning SRTTU – A. Akhavan Lecture 10 12 ۱۳۹۷ آﺒﺎﻥ ۲۶ ، ﺷﻨﺒﻪ