Clickstream Models Sybil Detection Gang Wang UC Santa

Clickstream Models & Sybil Detection Gang Wang (王刚) UC Santa Barbara gangw@cs. ucsb. ed

Modeling User Clickstream Events User-generated events � E. g. profile load, link follow, photo browse, friend invite � Assume we have event type, user. ID, timestamp User. ID Event Generated Timestamp Intuition: Sybil users act differently from normal users � Sybil users act differently from normal users Goal-oriented: focus on specific actions, less “extraneous” events Time-limited: focused on efficient use of time, smaller gaps?

System Overview 3 Sequence Clustering Clickstream Log Cluster Coloring Known Good Users Incoming Clickstream ? Legit Sybils

Clickstream Models 4 Clickstream log � user clicks (click type) with timestamp Modeling Clickstream � Event-only e. g. Sequence Model: order of events ABCDA � Time-based e. g. {t 1, t 2, t 3, …} � Hybrid e. g. Model: sequence of inter-arrival time Model: sequence of click events with time A(t 1)B(t 2)C(t 3)D(t 4)A

Clickstream Clustering 5 Similarity Graph � Vertices: users (or sessions) � Edges: weighted by the similarity score of two user’s clickstream Clustering Similar Clickstreams together � Graph partitioning using METIS Q: How to compare two clickstreams?

Distance Functions Of Each Model 6 Click Sequence (CS) Model � Ngram overlap S 1= AAB S 2= AAC � Ngram+count S 1= AAB S 2= AAC Euclidean Distance ngram 1= {A(2), B(1), AA(1), AB(1), AAB(1)} V 1=(2, 1, 0, 1, 1, 0)/6 ngram 2= {A(2), C(1), AA(1), AC(1), AAC(1)} V 2=(2, 0, 1, 1, 1, 0, 0, 1)/6 Time-based Model � � ngram 1= {A, B, AA, AB, AAB} ngram 2= {A, C, AA, AC, AAC} Compare the distribution of inter-arrival time K-S test Hybrid Model � � Bucketize inter-arrival time Compute 5 grams (similar with CS Model)

Detection In A Nutshell 7 Inputs: ? � Trained clusters � Input sequences for testing Methodology: given a test sequence A �K nearest neighbor: find the top-k nearest sequences in the trained cluster � Nearest Cluster: find the nearest cluster based on average distance to sequences in the cluster � Nearest Cluster (center): pre-compute the center(s) of cluster, find the nearest cluster center

Clustering Sequences 8 How well can each method separate Sybils from legitimate users? Model (Sequence Type) Distance Function (False positives, False negatives) of users 20 clusters 50 clusters 100 clusters (3% , 6%) (1%, 7%) (2%, 4%) (1%, 3%) 10 gram (1%, 3%) (2%, 2%) 10 gram+coun t (1%, 4%) (2%, 4%) (1%, 2%) Time-based Model K-S Test (9%, 8%) (2%, 10%) (5%, 10%) Hybrid Model 5 gram (3%, 2%) (2%, 2%) Click Sequence unigram Model (Categories) unigram+cou nt

Detection Accuracy 9 Basics � � Training on one group of users, and test on the other group of users. Clusters trained using Hybrid Model Key takeaways � � High accuracy with 50 clicks in the test sequence Nearest Cluster (Center) method achieves high accuracy with minor computation overhead Number of Clicks in the Sequence (length) (False positives, False negatives) of users K-nearest Neighbors (k=3) Nearest Cluster (Avg. Distance) Nearest Cluster (Center) Length <=50 (1. 5% , 2. 1%) (0. 6%, 2. 6%) (0. 4%, 2. 3%) Length <=100 (0. 9% , 1. 8%) (0. 2%, 2. 5%) (0. 3%, 2. 3%) (0. 6% , 3%) (0. 4%, 2. 8%) (0. 4%, 2. 3%) All

Can Model Be Effective Over Time? 10 Experiment method � Using first two-week data to train the model � Testing on the following two-week data Model Click Sequence Model Hybrid Model (False positives, False negatives) of users K-nearest Neighbors (k=3) Nearest Cluster (Avg. Distance) Nearest Cluster (Center) (1. 8% , 1%) (3%, 2%) (3%, 0. 8%) (3% , 2%) (3%, 1%) (1. 2%, 1. 4%)

Still Ongoing Work With broad interest and applications As Sybil detection tool � Code being tested internally at Renren Trained with 10 K users (2 -week log) Testing on 1 Million users (1 -week log) 5 Sybil clusters Further 22 K suspicious profiles improvement Training with longer clickstream (half users have <5 clicks in 2 week) More conservative in labeling Sybil clusters. As user modeling tool � Code being tested by Linked. In as user profiler

Some Useful Tools Graph Partitioning � Metis http: //glaros. dtc. umn. edu/gkhome/metis/overview Community Detection � Louvain code https: //sites. google. com/site/findcommunities/

Other Ongoing Works/Ideas Fighting against crowdturfing � Crowdturfing: real users are paid to spam � How to detect these malicious real users User behavior model Network-wised temporal anomaly detection Information Dissemination � Content sharing visa social edges How often will user click on the content How often will user comment on the content � Sybil detection, target ad placement

Thank You! Questions? http: //current. cs. ucsb. edu