Characterizing the Splogosphere Tim Finin Pranam Kolari Akshay

Characterizing the Splogosphere Tim Finin Pranam Kolari, Akshay Java and Tim Finin University of Maryland, Baltimore County 3 rd Annual Workshop on the Weblogging Ecosytem: Aggregation, Analysis and Dynamics 22 May 2006 UMBC an Honors University in Maryland http: //ebiquity. umbc. edu/paper/html/id/299/

Outline • Introduction • Motivation • Blog. Pulse Dataset • Weblogs. com Dataset • Implications UMBC an Honors University in Maryland

The Blogosphere • 57% of online US teens generate content, 40% read blogs, 20% have them! (Pew Nov. 2005) • 53% of companies are blogging (Guideware Oct. 2005) • My. Space accounts for 1/3 of all web clicks (Hendler, 2006) ? ! • But … the Blogosphere is awash in spam UMBC an Honors University in Maryland Source: Wikipedia

Blogosphere/Splogosphere UMBC an Honors University in Maryland

Spam in the Blogosphere • Types: comment spam, ping spam, spam blogs • Akismet: “ 87% of all comments are spam” • 75% of update pings are spam (ebiquity 2005) • 20% of indexed blogs by popular blog search engines is spam (Umbria 2006, ebiquity 2005) • “Spam blogs, sometimes referred to by the neologism splogs, are weblog sites which the author uses only for promoting affiliated websites” • “Spings, or ping spam, are pings that are sent from spam blogs” UMBC an Honors University in Maryland 1 Wikipedia

Motivation: host ads UMBC an Honors University in Maryland

Motivation: index affiliates, promote page. Rank UMBC an Honors University in Maryland

Spings from weblogs. com UMBC an Honors University in Maryland

Where do Splogs come from? “Honestly, Do you think people who make $10 k/month from adsense make blogs manually? Come on, they need to make them as fast as possible. Save Time = More Money! It's Common SENSE! How much money do you think you will save if you can increase your work pace by a hundred times? Think about it…” “Discover The Amazing Stealth Traffic Secrets Insiders Use To Drive Thousands Of Targeted Visitors To Any Site They Desire!” “Holy Grail Of Advertising. . . “ $ 197 UMBC an Honors University in Maryland “Easily Dominate Any Market, Any Search Engine, Any Keyword. ”

UMBC an Honors University in Maryland

Our splog bait was picked up and used by dozens of sploggers UMBC an Honors University in Maryland

UMBC an Honors University in Maryland

Our feed is RSSjacked by at least one splogger UMBC an Honors University in Maryland

Why are splogs a problem? • Splogs undermine ranking algorithms • Splogs water down search results • Splogs threaten the Web advertising model • Splogs indulge in “plagiarism” • Splogs skew results of market research tools • Splogs stress the Blogosphere infrastructure of ping servers, blog search engines, etc. UMBC an Honors University in Maryland

Outline • Introduction • Motivation • Blog. Pulse Dataset • Weblogs. com Dataset • Implications UMBC an Honors University in Maryland

Splog Detection • SVM based probabilistic splog detection (Kolari et al. , 2006) • Hand verified training set of blogs and splogs • Precision/Recall of 87% • Bag-of-words based feature using text on blog home-page, O(x) • Some additional local features P( x is a splog | O(x) ) P( x is a blog | O(x) ) UMBC an Honors University in Maryland top features blogs we what was my org flickr paper 600 open words weblog motion me thank go january trackback archives now political splogs find info news your 27 another website best articles on perfect products uncategorized 280 hot resources inc 60 three copyright

Outline • Introduction • Motivation • Blog. Pulse Dataset • Weblogs. com Dataset • Implications UMBC an Honors University in Maryland

Blog. Pulse Dataset • • 21 days of July 2005 1. 3 million blogs Eliminated Live-Journal Re-fetched blog-homepages, many spam blogs were nonexistent since spam blogs are short lived Arrived at 500 K samples Set probability thresholds to 0. 2 (authentic blog) and 0. 8 (splog) Identified 27 K splogs Sampled for 27 K authentic blogs UMBC an Honors University in Maryland

blogs and splogs Splogs vs. Blogs – Word Count blogs UMBC an Honors University in Maryland splogs

Splogs vs. Blogs – In-degree Top 5 http: //www. engadget. com 1942 http: //www. huffingtonpost. com/theblog 905 http: //www. crooksandliars. com 637 http: //blogs. guardian. co. uk/news 616 http: //www. littlegreenfootballs. com/weblog 611 Top 5 http: //spaces. msn. com/members/pony-girl 505 http: //spaces. msn. com/members/black-puss 505 http: //spaces. msn. com/members/amputee-women 505 http: //spaces. msn. com/members/free-stories 505 http: //spaces. msn. com/members/first-time-girl 505 UMBC an Honors University in Maryland

Splogs vs. Blogs – Out-degree Top 5 http: //www. xanga. com/home. aspx? user=hit_me_layoutz 273 http: //www. xanga. com/home. aspx? user=i_jock_layouts 271 http: //www. xanga. com/home. aspx? user=slp_layouts_slp 198 http: //spaces. msn. com/members/cyrustse 1986 193 http: //www. xanga. com/home. aspx? user=layouts_n_codes 2005 180 Top 5 http: //worldseriesofpokerchipscardguard. blogspot. com 898 http: //rule-wsop. blogspot. com 898 http: //worldseries-ofpoler. blogspot. com 898 http: //qsopcom-1. blogspot. com 898 http: //weopcom. blogspot. com 898 UMBC an Honors University in Maryland

Outline • Introduction • Motivation • Blog. Pulse Dataset • Weblogs. com Dataset • Implications UMBC an Honors University in Maryland

Weblogs. com Dataset • 20 Nov 2005 – 11 Dec 2005 • 16 million update pings • Pings subdivided by language: da, de, en, es, fi, fr, it, nl, pt, sv • Heuristics to identify Japanese, Chinese, Korean • Set threshold of 0. 5 to separate out authentic blogs from splogs. UMBC an Honors University in Maryland 1 Thanks to James Mayfield, JHU APL

Ping times – Italian Blogs UMBC an Honors University in Maryland

Sping vs. Ping times UMBC an Honors University in Maryland

Spings vs. Pings: frequency blogs vs. their ping frequency follows a power law, but splogs vs. spings does not UMBC an Honors University in Maryland

All Pings – 16 Million • Close to 40% spings • Among English blogs – 75% pings are spings –Authentic blogs are 13% of all pings • Including Info domain – 50% of all pings are spings UMBC an Honors University in Maryland url count http: //www. wiccapaganblog. com 1491 http: //www. freecancerfacts. com/wp 1452 http: //www. myaquariumiplace. com 1375 http: //www. criss-angel. biz 1215 http: //www. microdermabrasionsecrets. com 1211 http: //www. tipstohealth. com/blog 1207 http: //www. countrymusicdigest. com 1191

Outline • Introduction • Motivation • Blog. Pulse Dataset • Weblogs. com Dataset • Implications UMBC an Honors University in Maryland

Implications (1) • Blog. Pulse dataset – Local word models most effective for fast splog detection – If splogs escape filters, in-link and out-link distribution point to link-based classification • Weblogs. com dataset – Ping frequency can be useful – Splogs probably not a big problem in most European languages. Yet. • The nature of the domain, points to spam filters employing a multi-step, and adaptive approach, which we are currently pursuing UMBC an Honors University in Maryland

Implications (2) – Filter Design Spam Blog Filter Heuristics 1 Blog Identifier Language Identifiers 2 3 IP Blacklists Supporting Info (OPTIONAL) Authentic Blogs Spam Blogs UMBC an Honors University in Maryland Spam Blog Detectors 4

Conclusions • Blog spam is a serious problem – Classic arms race, e. g. , increased plagiarism, feedjacking • Blog spam identification requires different tactics than used for email and Web spam – Local features effective, but not sufficient – Lots of relational features (e. g. , links, ads, IP addresses, tight but disconnected communities) but dynamism reduces effectiveness of analysis • Getting good training sets expensive, especially in a multilingual environment. – Minute or more a judgment • Good opportunities for infrastructure insertion, e. g. , sping free ping servers UMBC an Honors University in Maryland

For more information http: //ebiquity. umbc. edu/ Annotated in OWL UMBC an Honors University in Maryland

Questions? UMBC an Honors University in Maryland

Blogs – A Specialized Domain Update Pings Ping Stream 2 3 Update Stream Update Pings 1 Fetch Content 1 UMBC an Honors University in Maryland (2 3 ) 4 4