Yelp Its AP Final Presentati on Aldred Lau

Yelp! It’s AP Final Presentati on Aldred Lau Wen Yang | Ng Hui Ying | Michelle Teo Sok Lee | Tan Yi Hao

Ratings show IF…

Reviews show WHY…

Content 1. 2. 3. 4. 5. Project Overview Analysis Process Visualizations Demo Insights and Recommendations Project Analysis Visualizations Demo Insights

PROJECT OVERVIEW - QUICK RECAP

Project Focus: Text and sentiment analysis on review texts for strategic business improvements Project Analysis Visualizations Demo Insights

Objectives of Project Distinguishi ng factors Observable trends or changes Desirable Traits Project Analysis Visualizations Demo Insights

Project Summary Yelp Text & Sentiment Analysis Explorati on • EDA • Secondary Research • Tooling Project 1 st Improving 2 nd Deliverabl Accuracy Deliverabl e e • Building • Scoping • Text & Sentiment Analysis Classifiers • Train Data • Comparing Results Visualizations Demo • Analysis Run 2 • Attribute Grouping & Scoring Insights

Before Mid Terms Initial Data Set Filter Busine ss Data Merge Busine ss + Review Data Project Analysis Split further (+pos/ne g) (+biz rating) Split further Text Analysis Split into States Visuali ze (+Biz. Cat) Visualizations Demo Insights

Mid Term Dashboard Project Analysis Visualizations Demo Insights

Post Mid Term • Things we worked on: Increasing Sentiment Accuracy Grouping Features into Attributes Rating Attributes Identifying Trends over Time 2 nd Dashboard Insights & Recommendations Project Analysis Visualizations Demo Insights

1 • Process to answer questions 2 • Scoping Process 3 • Text & Sentiment Analysis Process • 2 Run of Text & ANALYSIS PROCESS 4 nd Sentiment Analysis

Process of Answering Questions Common Desirable Attributes Scope Data Clean Text Data Generate Test & Training Data Extract Keywords & ngrams Visualize Test Against Test Data Classify & Score Data Attributes Filter Data (frequency) For Distinguishing Factor Project Analysis Visualizations Demo Insights

Process of Answering Questions Distinguishing Factor (Good VS Average VS Bad) • Compare attribute category scores over the various business types (good, average, bad) Project Analysis Visualizations Demo Insights

Process of Answering Questions Trends and Changes Over Time • Plotting of geospatial data of business categories, and business names over time Project Analysis Visualizations Demo Insights

Scoping Process Data Scoping (Refresher) Initial Data Set Locations • 4 Countries • 10 States Project Analysis Businesses Reviews & Ratings • 61184 business • 789 different business categories Visualizations • Stars (1 -5) • Review Text Demo Insights

Scoping Process Scoping (Filter Business Data) Removed non-US states & US states with little data Removed nonrestaurant businesses Selected 5 business categories Project Filtered Down Business Data Analysis Visualizations Demo Powered by: Insights

Scoping Process Scoping (Merge Business and Review) Filtered Down Business Data Review Text & Ratings Data for Text Analys is Powered by: Project Analysis Visualizations Demo Insights

Scoping Process Scoping (Split into States) State AZ State WI State IL Data for Text Analysi s State SC State PA State MA State NC State NV Project Analysis Powered by: Visualizations Demo Insights

Scoping Process Scoping (Split into State_Biz. Cat) Bakeries Dessert State Data for Text Analysis Coffee and Tea Project Bars Analysis Breakfas t and Brunch Visualizations Powered by: Demo Insights

Scoping Process Scoping (Split State_Biz. Cat by Sentiment) State_Biz. Cat Data for Text Analysis Positive State_Biz. Cat Data for Text Analysis Negative Project Analysis Visualizations Powered by: Demo Insights

Scoping Process Scoping (Further Split by Rating Score) State_ Biz. Cat_ Sentiment Data Project Analysis Good (>=4. 0) Bad (<=2. 5) Visualizations Powered by: Demo Insights

Scoping Process Scoping Summary Initial Data Set Split into States Filter Business Data Merge Business + Review Data for Text Analysi s Split further (+Biz. Cat) Split further (+pos/neg) (+biz rating) Powered by: Project Analysis Visualizations Demo Insights

Text & Sentiment Analysis Process Text Cleaning And Filtering Text Cleaning Remove nr Convert all text to lowercase Convert double spacing to single spacing Project Analysis Remove irrelevant words Remove words <3 char Filtered Text Data Visualizations Demo Insights

Text & Sentiment Analysis Process Extracting Keywords and Phrases Keywords Using NLTK library & Text. Blo b Ngram Word Bigram Adjective Trigram Noun Project Analysis Visualizations Powered by: Demo Insights

Text & Sentiment Analysis Process Filter By Frequency Count Top 20 keywords Using NLTK library & Text. Blo b ngram Word Bigram Adjective Trigram Noun Project Analysis Visualizations Powered by: Demo Insights

Text & Sentiment Analysis Process Sentiment Analysis Positive Using NLTK library & Text. Blo b Negative Project Analysis Visualizations Demo Insights

2 nd Run Text Analysis Classifiers – For Sentiment Analysis PA state data Extract 1000 words per category Rated positive/negative Tried adding 1000 manual classifiers Accuracy 71% (ours) VS 79% (in-built) Tested on 500 records (100/cat from other state) Tried adding manual classifiers with in-built Accuracy decreased Continued with in-built classifiers Project Analysis Visualizations Demo Insights

2 nd Run Text Analysis Attributes Categorization And Scoring Identify common themes among bigrams Manuall y Categorize into themes (Taste, service, ambience, variety, accessibility) Rated text within categories Project Analysis Visualizations Demo Insights

2 nd Run Text Analysis Attributes Categorization And Scoring Manually Categorized into Themes Project Analysis Rated Within Themes (1 -3) Visualizations Demo Insights

2 nd Run Text Analysis Re-Splitting and Merging of Data Trigram Project Added Business Name Analysis Visualizations Demo Insights

VISUALIZATION

Visualization (Desired Common Attributes) ❶ Word Cloud to identify commonly mentioned attributes. Positive words in green to identify desired common Project Analysis attributes. Insights Visualizations Demo

Visualization (Desired Common Attributes) ❶ Word Cloud ❷ Bar Chart Project Analysis Visualizations Demo Insights

Visualization (Desired Common Attributes) Word Cloud to identify commonly mentioned attributes. Positive words in green to identify desired common attributes. Project Analysis Visualizations Demo Insights

Visualization (Distinguishing Factors) Side-by-side word cloud and bar chart for analysis between 2 businesses Project Analysis Visualizations Demo Insights

Visualization (Trends OR Changes) ❶ Geospatial Visualization ❷ Bar Chart ❸ Line Chart Project Analysis Visualizations Demo Insights

Visualization (Trends OR Changes) 1. Geospatial Map Plot • Visual of business categories in location Project Analysis Visualizations Demo Insights

Visualization (Trends OR Changes) 2. Bar Chart • Visual of # businesses per business category Project Analysis Visualizations Demo Insights

Visualization (Trends OR Changes) 3. Line Chart • Visual of positive VS negative over time Project Analysis Visualizations Demo Insights

DEMO

INSIGHTS AND FUTURE ANALYSIS

Findings • Bigrams analysis only provided actionable outcomes for ambience and service attributes. • Trigram were generated but were even less helpful compared to bigrams due to low match rate to reviews Project Analysis Visualizations Demo Insights

Future Analysis • Automating the categorization & rating process • Generate more accurate classifier by creating own or by harnessing the power of crowd sourcing • Feature analysis to recommend food items to users Project Analysis Visualizations Demo Insights

THE END