Opinion Mining and Sentiment Analysis Slides from Bing

Opinion Mining and Sentiment Analysis Slides from Bing Liu and Ronan Feldman

Introduction l Two main types of textual information. § Facts and Opinions • Note: factual statements can imply opinions too. Most current text information processing methods (e. g. , web search, text mining) work with factual information. l Sentiment analysis or opinion mining l § computational study of opinions, sentiments and emotions expressed in text. l Why opinion mining now? Mainly because of the Web; huge volumes of opinionated text.

Introduction – user-generated media l Importance of opinions: § Opinions are useful when making a decision, we want to hear others’ opinions. § In the past, • Individuals: opinions from friends and family • businesses: surveys, focus groups, consultants … l Word-of-mouth on the Web § User-generated media: One can express opinions on anything in reviews, forums, discussion groups, blogs. . . § Opinions of global scale: No longer limited to: • Individuals: one’s circle of friends • Businesses: Small scale surveys, tiny focus groups, etc.

Sentiment analysis applications l Businesses and organizations § Benchmark products and services; market intelligence. • Businesses spend a huge amount of money to find consumer opinions using consultants, surveys and focus groups, etc l Individuals § Make decisions to purchase products or to use services § Find public opinions about political candidates and issues l Ad placement: e. g. in social media § Place an ad if one praises a product. § Place an ad from a competitor if one criticizes a product. l Opinion retrieval: provide general search for opinions.

A Fascinating Problem! l Intellectually challenging & major applications. § A popular research topic in recent years in NLP and Web data mining. § 20 -60 companies in USA alone l It touches every aspect of NLP and yet is restricted and confined. § Little research in NLP/Linguistics in the past. l Potentially a major technology from NLP § But “not yet” and not easy! § Data sourcing and data integration are hard too!

Abstract Problem Statement It consists of two parts 1. Opinion definition l § 2. What is an opinion? Opinion summarization § Opinions are subjective. An opinion from a single person (unless a VIP) is often not sufficient for action. § We need opinions from many people, and thus opinion summarization.

An Example Review l “I bought an i. Phone a few days ago. It was such a nice phone. The touch screen was really cool. The voice quality was clear too. Although the battery life was not long, that is ok for me. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive, and wanted me to return it to the shop. …” l What do we see? § Opinions, targets of opinions, and opinion holders

Entity and aspect/feature level l Id: Abc 123 on 5 -1 -2008 “I bought an i. Phone a few days ago. It is such a nice phone. The touch screen is really cool. The voice quality is clear too. It is much better than my old Blackberry, which was a terrible phone and so difficult to type with its tiny keys. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive …” l What do we see? § § Opinion targets: entities and their features Sentiments: positive and negative Opinion holders: persons who hold the opinions Time: when opinions were expressed

Two main types of opinions l Regular opinions: Sentiment/opinion expressions on some target entities § Direct opinions: “The touch screen is really cool” § Indirect opinions: “After taking the drug, my pain has gone” l Comparative opinions: Comparisons of more than one entity E. g. , “i. Phone is better than Blackberry” l We focus on regular opinions first, and just call them opinions.

Entity and Aspect l Definition (entity): An entity e is a product, person, event, organization, or topic. e is represented as § a hierarchy of components, sub-components, and so on. § Each node represents a component and is associated with a set of attributes of the component. An opinion can be expressed on any node or attribute of the node. l For simplicity, we use the term aspects (features) to represent both components and attributes. l B. Liu, Handbook of NLP, 2010

Direct Opinion: definition l A direct opinion is a quintuple (oj, fjk, sijkl, hi, tl), where § oj is a target object § fjk is a feature of the object oj § sijkl is the sentiment value of the opinion holder hi on feature fjk of object oj at time tl. sijkl is positive, negatve, or neutral, or a more granular rating § hi is an opinion holder § tl is the time when the opinion is expressed B. Liu, Handbook of NLP, 2010

Our example in quintuples (i. Phone, GENERAL, +, Abc 123, 5 -1 -2008) l (i. Phone, touch_screen, +, Abc 123, 5 -1 -2008) l

Alternative terminology Entity is also called object. l Aspect is also called feature, attribute, facet, etc l Opinion holder is also called opinion source l

Structure the unstructured l Goal: Given an opinionated document § Discover all quintuples (oj, fjk, sijkl, hi, tl) • i. e. , mine the five corresponding pieces of information in each quintuple, and § Or, solve some simpler problems • E. g. classify the sentiment of the entire document l With the quintuples, § Unstructured Text Structured Data • Traditional data and visualization tools can be used to slice, dice and visualize the results in all kinds of ways • Enable qualitative and quantitative analysis.

Sentiment Classification: doc-level l Classify a document (e. g. , a review) based on the overall sentiment expressed by opinion holder § Classes: Positive, or negative (and neutral) • In the model, (oj, fjk, sijkl, hi, tl), • It assumes – Each document focuses on a single object and contains opinions from a single opinion holder. – It considers opinion on the object, oj (or oj = fjk) Pang and Lee, et al 2002 and Turney 2002

Subjectivity l Sentence subjectivity: An objective sentence presents some factual information, while a subjective sentence expresses some personal opinions, beliefs, views, feelings, or emotions. § Not the same as emotion

Subjectivity Analysis l Sentence-level sentiment analysis has two tasks: § Subjectivity classification: Subjective or objective. • Objective: e. g. , I bought an i. Phone a few days ago. • Subjective: e. g. , It is such a nice phone. § Sentiment classification: For subjective sentences or clauses, classify positive or negative. • Positive: It is such a nice phone. l However (Liu, NLP handbook) § subjective sentences ≠ +ve or –ve opinions • E. g. , I think he came yesterday. § Objective sentence ≠ no opinion • Imply –ve opinion: My phone broke in the second day. Wiebe et al 2004

Rational and emotional evaluations l Rational evaluation: Many evaluation/opinion sentences express no emotion § E. g. “The voice on this phone is clear” l Emotional evaluation § E. g. “I love this phone” § “The voice on this phone is crystal clear” (? ) l Some emotion sentences express no (positive or negative) opinion/sentiment § E. g. “I am so surprised to see you”

Feature-Based Sentiment Analysis l Sentiment classification at both document and sentence (or clause) levels are not sufficient, § they do not tell what people like and/or dislike § A positive opinion on an object does not mean that the opinion holder likes everything. § An negative opinion on an object does not mean …. . l Objective: Discovering all quintuples (oj, fjk, soijkl, hi, tl) l With all quintuples, all kinds of analyses become possible.

Opinion Summary l With a lot of opinions, a summary is necessary. § A multi-document summarization task l For factual texts, summarization is to select the most important facts and present them in a sensible order while avoiding repetition § 1 fact = any number of the same fact l But for opinion documents, it is different because opinions have a quantitative side & have targets § 1 opinion ≠ a number of opinions l Aspect-based summary is more suitable § Quintuples form the basis for opinion summarization

Feature-Based Opinion Summary “I bought an i. Phone a few days ago. It was such a nice phone. The touch screen was really cool. The voice quality was clear too. Although the battery life was not long, that is ok for me. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive, and wanted me to return it to the shop. …” Hu & Liu, KDD-2004 Feature Based Summary: Feature 1: Touch screen Positive: 212 l The touch screen was really cool. l The touch screen was so easy to use and can do amazing things. … Negative: 6 l The screen is easily scratched. l I have a lot of difficulty in removing finger marks from the touch screen. … Feature 2: battery life … Note: We omit opinion holders

Visual Comparison n + Summary of reviews of Cell Phone 1 _ Voice n Comparison of reviews of + Cell Phone 1 Cell Phone 2 _ Liu et al. WWW-2005 Screen Battery Size Weight

Aspect-based opinion summary

Google Product Search

Comparing 3 GPSs on different features l Each bar shows the proportion of +ve opinion

Demo 1: Detail opinion sentences You can click on any bar to see the opinion sentences. Here are negative opinion sentences on the maps feature of Garmin. l The pie chart gives the proportions of opinions. l

# of feature mentions l People talked more about prices than other features. They are quite positive about price, but not bout maps and software. 27

Aggregate opinion trend l More complains in July - Aug, and in Oct – Dec! 28

Sentiment Analysis is Challenging! l “This past Saturday, I bought a Nokia phone and my girlfriend bought a Motorola phone with Bluetooth. We called each other when we got home. The voice on my phone was not so clear, worse than my previous phone. The battery life was long. My girlfriend was quite happy with her phone. I wanted a phone with good sound quality. So my purchase was a real disappointment. I returned the phone yesterday. ”

Requires solving several IE problems l (oj, fjk, sijkl, hi, tl), § § § oj - a target object: Named Entity Extraction (more) fjk - a feature of oj: Information Extraction sijkl is sentiment: Sentiment determination hi is an opinion holder: Information/Data Extraction tl is the time: Data Extraction Co-reference resolution Relation extraction Synonym match (voice = sound quality) … l None of them is a solved problem! l l l

Easier and harder problems l Tweets from Twitter are probably the easiest § short and thus usually straight to the point § Stocktwits are much harder! l Reviews are next § entities are given (almost) and there is little noise l Discussions, comments, and blogs are hard. § Multiple entities, comparisons, noisy, sarcasm, etc Extracting entities and aspects, and determining sentiments/opinions about them are hard. l Combining them is harder. l

Extraction of competing objects l l The user first gives a few objects/products as seeds, e. g. , BMW and Ford. The system then identifies other competing objects from the opinion corpus. The problem can be tackled with PU learning (Learning from positive and unlabeled examples) (Liu et al. , 2002, 2003). See Li et al. , ACL-2010

Feature/Aspect-based sentiment analysis

Aspect-based sentiment analysis Much of the research is based on online reviews l For reviews, aspect-based sentiment analysis is easier because the entity (i. e. , product name) is usually known l § Reviewers simply express positive and negative opinions on different aspects of the entity. l For blogs, forum discussions, etc. , it is harder: § both entity and aspects of entity are unknown, § there may also be many comparisons, and § there is also a lot of irrelevant information.

Find entities Although similar, it is somewhat different from the traditional named entity recognition (NER). l E. g. , one wants to study opinions on phones given Motorola and Nokia, find all phone brands and models in a corpus, e. g. , Samsung, Moto. l

Feature/Aspect extraction l Extraction may use: § frequent nouns and noun phrases • Sometimes limited to a set known to be related to the entity of interest or using part discriminators • e. g. , for a scanner entity “of scanner”, “scanner has” l opinion and target relations § Proximity or syntactic dependency l Standard IE methods § Rule-based or supervised learning § Often HMMs or CRFs (like standard IE)

Double Propagation l Proposed in Qiu et al. , IJCAI-2009 § Like co-training l It exploits the dependency relations of opinions and features to extract features. § Opinions words modify object features, e. g. , § “This camera takes great pictures” l The algorithm bootstraps using a set of seed opinion words (no feature input). § To extract features (and also opinion words)

The DP method l DP is a bootstrapping method § Input: a set of seed opinion words, § no aspect seeds needed l Based on dependency tree (Tesniere 1959) This phone has good screen

Rules from dependency grammar

Group synonym features Features that are domain synonyms should be grouped together. l Many techniques can be used to deal with the problem, e. g. , l § Topic modeling, distributional similarity, etc l Semi-supervised learning method § Z. Zhai, B. Liu, H. Xu and P. Jia. Grouping Product Features Using Semi-Supervised Learning with Soft. Constraints. COLING-2010.

Coreference Resolution l Different from traditional coreference resolution § Important to resolve objects and features § E. g. . , “I bought a Canon S 500 camera yesterday. It looked beautiful. I took a few photos last night. They were amazing”. l Some specific characteristics of opinions can be exploited for better accuracy. See § X. Ding and B. Liu, Resolving Object and Attribute Coreference in Opinion Mining. COLING-2010.

Coreference Resolution Attardi et al. 2010. Coreference Resolution by Parse Analysis and Similarity Clustering. l Best performing at Sem. Eval 2010 l § Identifies mentions from dependency trees § Uses eager classifier to cluster mentions Positive and negative instances are created by pairing each mention with each of the preceding ones l Features extracted from pairs of mentions: l Lexical: Same, Prefix, Suffix, Acronym … Distance: Edit, mention, token, sentence Syntax: same Head. Po. S, pair of Head. Pos Pair of counts of mention occurrences Same NE type For pronouns: type, pair of genders, pair of numbers

Accuracy at Sem. Eval 2010 Mention CEAF B 3 Catalan 82. 7 57. 1 64. 6 German 59. 2 49. 5 50. 7 English 73. 9 57. 3 61. 3 Spanish 83. 1 59. 3 66. 0

Coreference Resolution Method of (Lee, Peirsman et al. 2011) l Best-performing in Co. NLL-2011 l § Based on locating all noun phrases, identifying their properties, and then clustering them in several deterministic iterations (called sieves), starting with the highest-confidence rules and moving to lower-confidence higher-recall ones. § Eager approach: any matching noun phrases with matching properties are immediately clustered together.

Identify opinion orientation For each feature, we identify the sentiment or opinion orientation expressed by a reviewer. l Almost all approaches make use of opinion words and phrases. But notice again (a simplistic way): l § Some opinion words have context independent orientations, e. g. , “great”. § Some other opinion words have context dependent orientations, e. g. , “small” § Many ways to use opinion words. l Machine learning methods for sentiment classification at the sentence and clause levels are also applicable.

Aggregation of opinion words l l Input: a pair (f, s), where f is a product feature and s is a sentence that contains f. Output: whether the opinion on f in s is positive, negative, or neutral. Two steps: § Step 1: split the sentence if needed based on BUT words (but, except that, etc). § Step 2: work on the segment sf containing f. Let the set of opinion words in sf be w 1, . . , wn. Sum up their orientations (1, -1, 0), and assign the orientation to (f, s) accordingly. In (Ding et al, WSDM-08), step 2 is changed to with better results. wi. o is the opinion orientation of wi. Dingd(w and Liu, , f)2008 is the distance from f to w.

Basic Opinion Rules Opinions are governed by some rules, e. g. , 1. Negative 2. Positive 3. Negation Neg Positive 4. Negation Pos Negative 5. Desired value range Positive 6. Below or above the desired value range Negative B. Liu, NLP handbook, 2010

Basic Opinion Rules Decreased Neg Positive 8. Decreased Pos Negative 9. Increased Negative 10. Increased Positive 11. Consume resource Negative 12. Produce resource Positive 13. Consume waste Positive 14. Produce waste Negative 7. B. Liu, NLP handbook, 2010

Two Main Types of Opinions l Direct Opinions: direct sentiment expressions on some target objects, e. g. , products, events, topics, persons. § E. g. , “the picture quality of this camera is great. ” § (many are much more complex). l Comparative Opinions: Comparisons expressing similarities or differences of more than one object. Usually stating an ordering or preference. § E. g. , “car x is cheaper than car y. ”

Comparative Opinions l Gradable § Non-Equal Gradable: Relations of the type greater or less than • Ex: “optics of camera A is better than that of camera B” § Equative: Relations of the type equal to • Ex: “camera A and camera B both come in 7 MP” § Superlative: Relations of the type greater or less than all others • Ex: “camera A is the cheapest camera available in market” Jindal and Liu, 2006

Mining Comparative Opinions l Objective: Given an opinionated document d Extract comparative opinions: (O 1, O 2, F, po, h, t), where O 1 and O 2 are the object sets being compared based on their shared features F po is the preferred object set of the opinion holder h t is the time when the comparative opinion is expressed. l Note: not positive or negative opinions.

Sentiment Lexicon

Sentiment (opinion) lexicon Sentiment lexicon: lists of words and expressions used to express people’s subjective feelings and sentiments/opinions l sentiments/opinions. l § Not just individual words, but also phrases and idioms, e. g. § “costs an arm and a leg” l Many sentiment lexica can be found on the web § They often have thousands of terms, and are quite useful

Sentiment lexicon Sentiment words or phrases (also called polar words, l opinion bearing words, etc). E. g. , l § Positive: beautiful, wonderful, good, amazing, § Negative: bad, poor, terrible, cost an arm and a leg. Many of them are context dependent, not just application domain dependent. l Three main ways to compile such lists: l § Manual approach: not a bad idea for a one-time effort § Corpus-based approach § Dictionary-based approach

Corpus vs Dictionary-based method l Corpus-based approaches § Often use a double propagation between opinion words and the items they modify § require a large corpus to get good coverage l Dictionary-based methods § Typically use Word. Net’s synsets and hierarchies to acquire opinion words § usually do not give domain or context dependent meanings

Corpus-based approaches Rely on syntactic patterns in large corpora. l (Hazivassiloglou and Mc. Keown, 1997; Turney, 2002; Yu and Hazivassiloglou, 2003; Kanayama and Nasukawa, 2006; Ding, Liu and Yu, 2008) l § Can find domain dependent orientations (positive, negative, or neutral). (Turney, 2002) and (Yu and Hazivassiloglou, 2003) l are similar. l § Assign opinion orientations (polarities) to words/phrases. § (Yu and Hazivassiloglou, 2003) is slightly different from (Turney, 2002) • use more seed words (rather than two) and use log-likelihood ratio (rather than PMI).

The Double Propagation method The DP method can also use dependency of opinions & aspects to extract new opinion words. l Based on dependency relations l § Knowing an aspect can find the opinion word that modifies it • E. g. “The rooms are spacious” § Knowing some opinion words can find more opinion words • E. g. “The rooms are spacious and beautiful” l Jijkoun, Rijke and Weerkamp (2010) did similarly

Opinions implied by objective terms l Most opinion words are adjectives and adverbs, e. g. , good, bad, etc § There also many subjective and opinion verbs and nouns, e. g. , hate (VB), love (VB), crap (NN). l But objective nouns can imply opinions too § E. g. “After sleeping on the mattress for one month, a body impression has formed in the middle” l How to discover such nouns in a domain or context?

Pruning For an aspect with an implied opinion, it has a fixed opinion, either +ve or -ve, but not both. l We find two direct modification relations using a dependency parser. l § Type 1: O → O-Dep → A • This TV has good picture quality § Type 2: O → O-Dep → H ← A-Dep ← A § E. g. The springs of the mattress are bad l If an aspect has mixed opinions based on the two dependency relations, prune it.

Dictionary-based methods l Typically use Word. Net’s synsets and hierarchies to acquire opinion words § Start with a small seed set of opinion words. § Bootstrap the set to search for synonyms and antonyms in Word. Net iteratively (Hu and Liu, 2004; Kim and Hovy, 2004; Valitutti, Strapparava and Stock, 2004; Mohammad, Dunne and Dorr, 2009). l Kamps et al. , (2004) proposed a Word. Net distance method to determine the sentiment orientation of a given adjective

Semi-supervised learning (Esuli and Sebastiani, 2005) l Use supervised learning § Given two seed sets: positive set P, negative set N § The two seed sets are then expanded using synonym and antonymy relations in an online dictionary to generate the expanded sets P’ and N’ form the training sets l Using all the glosses in a dictionary for each term in P’ N’ and converting them to a vector l Build a binary classifier l Senti. Wordnet l

Which approach to use? Both corpus and dictionary based approaches are needed. l Dictionary usually does not give domain or context dependent meanings l § Corpus is needed for that l Corpus-based approach is hard to find a very large set of opinion words § Dictionary is good for that l In practice, corpus, dictionary and manual approaches are all needed.

Deep Analysis for Sentiment Analysis L’i. Phone è il mio preferito Android è preferito all’i. Phone Android è meno preferito dell’i. Phone Il gioco preferito per Android è l’obiettivo preferito dai pirati Lo schermo non è tanto bello

Syntax Tree ROOT PRED SUBJ MOD COMP PREP MOD Android è l’ obiettivo preferito dai pirati informatici

Deep Analysis Starts from syntax tree l Identifies mentions and relations l Applies filters l Assigns score l

Example CONJ PRED SUBJ PRED Il prezzo è elevato ma la qualità è notevole l Mention 1: il prezzo è elevato § Concept: prezzo § Attribute: elevato § Value: -1. 00 l Mention 2: la qualità è notevole § Concept: qualità § Attribute: elevato § Value: +4. 00

Spam Detection

Opinion Spam Detection n n Fake/untruthful reviews: n Write undeserving positive reviews for some target objects in order to promote them. n Write unfair or malicious negative reviews for some target objects to damage their reputations. Increasing number of customers wary of fake reviews (biased reviews, paid reviews) Jindal and Liu, 2007, 2008

An Example Practice of Review Spam Belkin International, Inc l Top networking and peripherals manufacturer | Sales ~ $500 million in 2008 l Posted an ad for writing fake reviews on amazon. com (65 cents per review) Jan 2009

Experiments with Amazon Reviews l June 2006 § 5. 8 mil reviews, 1. 2 mil products and 2. 1 mil reviewers. l A review has 8 parts • <Product ID> <Reviewer ID> <Rating> <Date> <Review Title> <Review Body> <Number of Helpful feedbacks> <Number of Feedbacks> <Number of Helpful Feedbacks> l Industry manufactured products “m. Products” e. g. electronics, computers, accessories, etc § 228 K reviews, 36 K products and 165 K reviewers.

Some Tentative Results Negative outlier reviews tend to be heavily spammed l Those reviews that are the only reviews of some products are likely to be spammed l Top-ranked reviewers are more likely to be spammers l Spam reviews can get good helpful feedbacks and non-spam reviews can get bad feedbacks l

Meeting Social Sciences l Extract and analyze political opinions. § Candidates and issues l Compare opinions across cultures and lang. § Comparing opinions of people from different countries on the same issue or topic, e. g. , Internet diplomacy l Opinion spam (fake opinions) § What are social, culture, economic aspects of it? Opinion propagation in social contexts l How opinions on the Web influence the real world l § Are they correlated? l Emotion analysis in social context & virtual world

Web. Says + Tiscali 17/1/2013

Monitoring Brexit Referendum Predicted by Web Analysis Exit Polls http: //www. sense-eu. info/

Summary l We briefly defined sentiment analysis problem. § Direct opinions: focused on feature level analysis § Comparative opinions: different types of comparisons § Opinion spam detection: fake reviews. • Currently working with Google (Google research award). A lot of applications. l Technical challenges are still huge. l § But I am quite optimistic. l Interested in collaboration with social scientists § opinions and related issues are inherently social.

References l B. Liu, “Sentiment Analysis and Subjectivity. ” A Chapter in Handbook of Natural Language Processing, 2 nd Edition, 2010. § http: //www. cs. uic. edu/~liub/FBS/sentiment-analysis. html