Towards Detecting Compromised Accounts on Social Networks Authors

Towards Detecting Compromised Accounts on Social Networks Authors: Manuel Egele, Gianluca Stringhini, Christopher Kruegel, and Giovanni Vigna Evan Sam Yuan

Introduction and Purpose A company can use its social media account help to create a trust with customers, and account breaches can break that trust. Account breaches can have significant fiscal impacts on the company, but they can also be detrimental to the stock market. Even after fake news has been removed and cleared up, there are often still lasting impacts on the company and the economy. Earlier detection and removal of unauthorized account usages will help minimize their effects.

Historical Social Network Compromises ● Associated Press ○ Distributed a news story about an explosion at the white house ● Fox News ○ Spread a story about Obama being assassinated ● Skype ○ Told followers that Microsoft was disclosing private information to government agencies ● Yahoo! ○ Spread a story about an Ebola outbreak in Atlanta ● Chipotle ○ Faked ‘compromised’ 10 tweets for publicity

COMPA ● Designed to detect compromised social network accounts. ● Detects unusual activity rather monitoring for all suspicious activity. ● Detects one-off attacks on high profile target accounts, as well as ‘campaign-attacks’ on a larger number of regular user accounts. ● Tested and verified on previously mentioned attacks.

Behavior Profiles To decide what counts as ‘normal behavior’, a behavior profile must be created for each user. This profile will be built off of all of the information that can be gathered from their public profile, called their ‘message stream’. For a good profile, the information stream must include messages from as many facets of interaction as possible. We need at least 10 posts/interactions to create a sufficient profile (any profiles with less than 10 posts will likely have little large-scale influence).

Profile Features ● ● ● ● Time of Day Message Source Language Topics Links Direct User Interaction Proximity

Model Training Input: series of messages (the message stream), at least 10 to create reliable profiles Output: each feature (f) model is represented as Mf = <fv, c> fv: value of a feature; c: count number of fv Mandatory models: there is one feature value for each message, and this feature is always present. [time of the day, source, proximity, language] Optional models: not every message has a value for the feature. [links, direct interaction, topic]

Special Training Rules: time of the day 1. Store for each hour the number of messages posted, as <i, ci> i: which hour; ci: count of messages posted within the ith hour 2. Adjustment step to relieve the impact of discretization For each <i, ci> tuple, replace it with <i, (ci-1+ci+ci+1)/3>

Classifying a New Message ● Basic rule: A message is considered anomalous if the value of a particular feature did not appear in history, or appeared only for a small number of times. ● Mandatory features: 1. If the feature value (fv) never appeared in Mf, anomalous-score = 1. 2. Find from Mf the corresponding tuple <fv, c>. If c is above average (i. e. often appears), anomalous-score = 0. Otherwise anomalous-score = 1 -c/N. ● Optional features: 1. If the feature value (fv) did appear in Mf, anomalous-score = 0. 2. Otherwise, anomalous-score = Cnull/N. Not related to the feature value in the examined message!?

Classifying a New Message Cont’d ● Final anomaly score: weighted sum of the value of all features. ● Training of weights: a training set of instances (messages and corresponding user histories) Twitter: 5236 messages (5142 legitimate, 94 malicious) Facebook: 279 messages (181 legitimate, 122 malicious) ● Problem: training set is not reliable. Twitter set is too biased. Facebook set is too small.

Behavioral Profile Stability ● Popular accounts are mostly well-maintained and should behave rather stably Exceptions (examining most recent 100 tweets): Twitter Account Name Violation Rate wordpress 28% Ask. Dot. Com 39% bookingcom 44% twitter 46% guardian 47%

Behavioral Profile Stability ● Regular accounts (normal users like most of us) are more likely to try something new, thus suffering more from false positives ● Solution: detect large-scale social network compromises instead of judging in the view of one single account

Detect Large-Scale Social Network Compromises Step 1: grouping messages. Content similarity: n-gram analysis => same 4 consecutive words URL similarity: contain links to similar URL(s) Limitation 1: discard facebook and youtube links Limitation 2: unable to analyze short URLs Step 2: checking all grouped messages for behavioral profile violations. Step 3: determine if the group is compromised on basis of the proportion of potentially compromised accounts; larger group more likely to be anomalous. Threshold(n) = max(0. 1, kn+d) [n: num of messages in the group] k=-0. 005 and d=0. 82: empirically determined by small-scale experiments

Handling Bulk Applications ● Some applications use templates to let user post similar message. ● Determination: for each application, randomly pick S messages and compute pairwise Levenshtein Ratios to see whether they are mostly similar. ● Discard groups that are due to popular bulk applications. Applications with lots of users and long history are considered safe.

Data Collection ● Twitter ○ ○ ○ Collected roughly 15 million tweets per day through the streaming API for ~ 3 months, netting over 1. 4 billion tweets. Historical activity for accounts with generated profiles was retrieved via the restful API; limited to 20, 000 API calls per hour (estimate ~360 million tweets) Twitter has rate limits for individual users, so messages were grouped (as discussed). ● Facebook ○ Dataset obtained from independent research group since Facebook does not offer convenient access to user data. (�� ) Specifically, the dataset came from geographic networks that group people who live in the same area (e. g. , London, NYC, LA). Researchers created FB profiles and joined these geographic networks to collect data.

Detection on Twitter ● Message text similarity yielded 374, 920 groups, 9, 362 of which were identified as compromised. Results based on URL similarity are also provided in paper. ● Those compromised groups were associated with 343, 229 accounts. ● 377 of the groups identified as compromised (4%) were false positives, corresponding to 12, 382 false positives for individiual accounts (3. 6%). ○ False positives attributed to short length in tweets used to build profile. ● False negatives: 64, 000 accounts were randomly sampled, resulting in an approximate rate of 4%.

Detection on Facebook ● Only used text similarity to create groups. COMPA generated 206, 876 profiles in 48, 586 groups; 671 groups were flagged as compromised (11, 499 accounts). ● False positives—a small number of applications (6, think Mafia Wars, etc) caused a large # of false positives so they were removed from the dataset. Enough posting to profile that it affected behavioral profile. With that in mind, 22 legit groups were incorrectly classified resulting in 3. 3% false positives. ● False negatives not discussed for Facebook dataset. This would have been even more challenging than Twitter considering data was from 2009.

Case Studies ● Associated Press ○ ○ ○ Fake news went through Twitter website, while AP typically uses 'Social. Flow' app Missing link to further info, uncharacteristic of AP tweets Further language analysis could've determined capitalization error ● Fox. News Politics ○ ○ Fake tweet violated nearly every feature Sent in the middle of the night, contained links and hash tags (atypical for account) ● Skype ○ ○ Like AP, most features were flagged except for language and time of posting This occurred during Skype's typical posting time, so why did it take them 2+ hours to pull the tweet (before it got 8000+ retweets)? Possibly because holiday season

Case Studies, Continued ● Yahoo! News ○ ○ Like with AP, fake tweet was sent through Twitter's website while Yahoo typically posts content from the 'Tweet. Deck' third-party app Fake tweet was also identified by lack of citing (using @) and external links to more information ● Chipotle ○ ○ The company pretended their account was compromised for a viral marketing campaign The only feature that was flagged by COMPA was that, unlike typical Chipotle tweets, the fake tweets were lacking in @-mentions. COMPA determined these tweets to be legitimate. A similar 'fake compromise' of MTV was correctly identified as well.

Limitations ● If an attacker is aware of COMPA, they could very easily post malicious content and go undetected by attempting to replicate the target's posting behavior. ○ ○ However, this also increases the risk of being detected and mitigated by the target. COMPA could be extended with additional features. ● Not discussed: privacy concerns associated with mining user activity to generate behavioral profiles. ○ Considering recent news, regulation will likely continue to increase.

Related Work ● Propogation of spam on Twitter: how do spammers leverage popular topics (trending hashtags) to maximize their reach? (Yardi et al. ) ● Monarch detects malicious messages on social networking sites by comparing URLs against known malicious websites. (Thomas et al. ) ● Warning. Bird analyzes HTTP redirection chains to determine if a given URL may eventually take users to a spam page. (Lee, Kim) ● Generally, there has been a shift from analyzing the interactions between users and toward the contents of messages coming from users.

Conclusion ● Summary ● Pros ○ ○ Simple method to solve large-scale problems Well design for dealing with special cases ● Cons ○ ○ No technical challenge, problems are simply neglected Training/testing samples are too biased, not enough malicious cases

Q&A