Rococo Extract more concurrency from distributed transactions Shuai

Rococo: Extract more concurrency from distributed transactions Shuai Mu, Yang Cui, Yang Zhang, Wyatt Lloyd, Jinyang Li Tsinghua University, New York University, University of Southern California, Facebook 1

What Large Web Sites Need 36. 8 million sold/day 170 million sold/day $169 billion trade/day Scalable Storage w/ Transactions! …… 2

What is a Distributed Transaction ？ BEGIN_TX if (iphone > 0) { iphone--; } if (case > 0) { case--; } Item i. Phone 6 Plus Stock 1 i. Phone Case 1 END_TX if (iphone > 0) { iphone--; } i. Phone=1 Case=1 if (case > 0) { case--; } Transactions should be strictly serializable! Otherwise… 3

Loss of serializability = angry customer 4

Serializability is Costly under Contention Throughput transaction/s Two-Phase Locking (2 PL) Optimistic Concurrency Control (OCC) # of concurrent transactions 5

OCC Aborts Contended Transactions if (iphone > 0) { iphone--; } if (case > 0) { case--; } Two-Phase Commit Execute (2 PC) if (iphone > 0) { iphone--; } if (case > 0) { case--; } If (iphone > 0) { iphone--; } i. Phone=1 Case=1 if (case > 0) { case--; } 6

2 PL Blocks Contended Transactions if (iphone > 0) { iphone--; } if (case > 0) { case--; } 2 PC Execute if (iphone > 0) { iphone--; } if (case > 0) { case--; } If (iphone > 0) { iphone--; } if (case > 0) { case--; } i. Phone=1 Case=1 if (case > 0) { case--; } 7

Achieve serializability w/o aborting or blocking* * for common workloads 8

Rococo’s Approach if (iphone > 0) { iphone--; } if (case > 0) { case--; } If (iphone > 0) { iphone--; } i. Phone=1 Case=1 if (case > 0) { case--; } Defer piece execution to enable reordering 9

Rococo Overview: Key techniques Enable piece for • Most pieces are executed at the secondreordering (commit) phase serializability 1. Two-phase protocol 2. Decentralized dependency tracking • Servers track pieces’ arrival order • Identify non-serializable orders • Deterministically reorder pieces Avoid 3. Offline workload checking aborts for common workloads • Identifies safe workloads (common) • Identifies small parts that need traditional approaches (rare) 10

#1 Two-phase protocol Start Phase Commit Phase Send pieces to servers w/o executing them Establish a final order and execute pieces Set up a provisional order Reorder for serializability 11

#2 Dependency Tracking: Start Phase T 1: if. . . iphone-if. . . case-- dep T 2 T 1 T 2: if. . . iphone-if. . . case-- dep if. . . iphone-- T 1 if. . . case-- T 2 if. . . iphone-- T 2 if. . . case-- T 1 i. Phone=1 Case=1 12

#2 Dependency Tracking: Commit Phase dep T 2 T 1 T 2 Sort the cycle by any T 2 deterministic order dep if. . . iphone-- T 1 T 1 if. . . case-- T 1 if. . . iphone-- T 2 if. . . case-- T 2 T 1 i. Phone=1 dep T 2 T 1 Case=1 13

Problem: Not Every Piece is Deferrable oid = next_oid ++ if. . . iphone-if. . . case-orderline. insert(oid, …) next_oid i. Phone Intermediate Results Calls for Immediate Execution Case orderline. insert( oid = next_oid ++ if. . . iphone-- if. . . case-- oid, ……) 14

Immediate Pieces are Naughty! a = next_a++; b = next_b++; ol_a. insert(a, …); ol_b. insert(b, …); next_a = 0 next_b = 0 a = next_a++ b = next_b ++ 1. Common workloads have few immediate pieces 15 2. Pre-known workload Offline check

#3: Offline Checking: Basic S(ibling)-edge linking pieces within a transaction T 1 An SC-cycle consists of T 2 both S-edge and C-edge Item_Tabl T 1 iphone e Item_Table case C(onflict)-edge Item_Tabl iphone e Item_Table case linking pieces w/ conflicting access SC-cycles represent potential non-serializable executions that require 2 PL/OCC 16

#3: Offline Checking: Enhanced T 1 Oid. Table Item. Tabl e Immediate T 2 Oid. Table Item. Tabl e Deferrable Item. Tabl e SC-cycles with deferrable pieces can be safely reordered! 17

Incorporating Immediate Pieces oid iphone Every cycle case contains deferrable pieces, ensured orderline by offline checking A cycle with case deferrable pieces is orderline safe to reorder oid iphone case orderline dep dep T 2 iphone T 1 Immediate dependency next_oid T 1 T 2 Deferrable i. Phone dependency T 2 T 1 Case T 1 T 2 orderline 18

Deferred Execution Decentralized Dependency Tracking Offline Checking Merged Pieces Read-only Transactions Reducing Dep. Size Fault Tolerance Overlapping Trans. 19

Evaluation • How does Rococo perform under contention? • How does Rococo scale? 20

Workload: Scaled TPC-C • One warehouse, many districts • Partitioned by districts - all transactions distributed New order (45%) Payment (43%) 5~15 5~15 Delivery (4%) Read-only: Order. Status(4%), Stock. Level (4%) 21

Throughput-(new-order/s) Rococo Has Higher Throughput 7000 6000 5000 4000 3000 Rococo Increasing graph size 2 PL OCC Aborts due to 2000 deadlock 1000 detection Aborts due to 0 0 50 100 conflicts in Concurrent reqs/server validation 8 servers, 10 districts per server Main source of 22 contention is next_oid of each district

Rococo Avoids Aborts 1, 2 61~66 ms Commit rate 1 Rococo 0, 8 2 PL 0, 6 OCC 0, 4 152~392 ms 0, 2 173~645 ms 0 0 20 40 60 80 Concurrent reqs/server 100 23

Rococo Scales Out Almost Linear Throughput-(new-order/s) 60000 50000 Rococo 40000 2 PL 30000 Blocking OCC 20000 10000 0 0 20 40 60 Number of servers 80 10 districts per server, fixed # of items contention grows slowly Aborts 24

Related Work Decentralized dependency tracking Isolation Level Concurrency Control Mech. Rococo Strict-Serial. Rococo Calvin [SIGMOD’ 12] Strict-Serial. Pre-ordered Locking Spanner [OSDI’ 12] Strict-Serial. 2 PL HStore [VLDB’ 07] Strict-Serial. OCC Centralized sequencing layer, Lynx [SOSP’ 13] Serial. Origin Ordering difficult to. Timestamp scale based Granola [SIGMOD’ 10] Serial. Percolator [OSDI’ 10] S. I. OCC Walter [SOSP’ 11] P. S. I. W-W Preclusion COPS [SOSP’ 11] Causal Dependency Tracking 25

Conclusion • Traditional protocols perform poorly w/ contention • OCC aborts & 2 PL blocks • Rococo defers execution to enable reordering • Strict serializability w/o aborting or blocking for common workloads • Rococo outperforms 2 PL & OCC • With growing contention • Scales out 26

Thank you! Questions? https: //github. com/msmummy/rococo Poster tonight! 27