Janus Consolidating Concurrency Control and Consensus for Commits

  • Slides: 20
Download presentation
Janus Consolidating Concurrency Control and Consensus for Commits under Conflicts Shuai Mu, Lamont Nelson,

Janus Consolidating Concurrency Control and Consensus for Commits under Conflicts Shuai Mu, Lamont Nelson, Wyatt Lloyd, Jinyang Li New York University, University of Southern California

State of the Art for Distributed Transactions Layer Concurrency Control on top of Consensus

State of the Art for Distributed Transactions Layer Concurrency Control on top of Consensus California Texas New York Paxos Transaction Protocol (e. g. , 2 PC) Shard for scalability Geo-replicate for fault tolerance Paxos

Latency Limitation: Multiple Wide-Area Round Trips from Layering California a++ b++ Texas New York

Latency Limitation: Multiple Wide-Area Round Trips from Layering California a++ b++ Texas New York

Throughput Limitation: Conflicts Cause Aborts California a++ b++ Texas New York a*=2 b*=2

Throughput Limitation: Conflicts Cause Aborts California a++ b++ Texas New York a*=2 b*=2

Goals: Fewer Wide-Area Round Trips and Commits Under Conflicts Best case wide-area RTTs 1

Goals: Fewer Wide-Area Round Trips and Commits Under Conflicts Best case wide-area RTTs 1 ≥ 2 Tapir [SOSP’ 15]. . . Spanner [OSDI’ 12]. . . Aborts Janus Calvin [SIGMOD’ 12]. . . Commits Behavior under conflicts

Establish Order Before Execution to Avoid Aborts • Designed for transactions with static read

Establish Order Before Execution to Avoid Aborts • Designed for transactions with static read & write-sets • Structure a transaction as a set of stored procedure pieces • Servers establishes consistent ordering for pieces before execution a++ b++ b*=2 a*=2 b++ a Challenge: Distributed ordering to avoid bottleneck b

Establish Order for Transactions and Replication Together to Commit in 1 Wide-area Roundtrip •

Establish Order for Transactions and Replication Together to Commit in 1 Wide-area Roundtrip • Consistent ordering for transaction and replication is the same! • Layering establishes the same order twice while Janus orders once Transaction a++ b++ a*=2 a a++ a' Replica of a a*=2 b*=2 b++ b Replication a++ Challenge: Fault tolerance for ordering

Overview of the Janus Protocol Pre-accept Send pieces to servers Establish initial order using

Overview of the Janus Protocol Pre-accept Send pieces to servers Establish initial order using dependencies Detect conflicts Conflicts? No Yes Accept Replicate dependencies Commit Establish final ordering Execute pieces in order

No Conflicts: Commit in 1 Wide-Area Round Trip Pre-accept Commit A Execute B Execute

No Conflicts: Commit in 1 Wide-Area Round Trip Pre-accept Commit A Execute B Execute Ne w Yo rk Ca l ifo rn ia 1 Local RTT A Execute B Execute 1 Wide-area RTT

Conflicts: Commit in 2 Wide-Area RTT A Ne w Yo rk Ca l ifo

Conflicts: Commit in 2 Wide-Area RTT A Ne w Yo rk Ca l ifo rn ia Pre-accept B A B ∪ Accept Commit

Conflicts: Commit in 2 Wide-Area Round Trips Accept A Ne w Yo rk Ca

Conflicts: Commit in 2 Wide-Area Round Trips Accept A Ne w Yo rk Ca l ifo rn ia Pre-accept B A B ∪ Commit

Conflicts: Commit in 2 Wide-Area Round Trip Commit Deterministically Order Cycles A Execute B

Conflicts: Commit in 2 Wide-Area Round Trip Commit Deterministically Order Cycles A Execute B Execute Ne w Yo rk Ca l ifo rn ia Accept Merge Dependencies A Execute B Execute

Janus Achieves Fewer Wide-Area Round Trips and Commits Under Conflicts • No conflicts: commit

Janus Achieves Fewer Wide-Area Round Trips and Commits Under Conflicts • No conflicts: commit in 1 wide-area round trip • Pre-accept sufficient to ensure same order under failures • Conflicts: commit in 2 wide-area round trips • Accept phase replicates dependencies to ensure same order under failures

Janus Paper Includes Many More Details • Full details of execution • Quorum sizes

Janus Paper Includes Many More Details • Full details of execution • Quorum sizes • Behavior under server failure • Behavior under coordinator (client) failure • Design extensions to handle dynamic read & write sets

Evaluation https: //github. com/NYU-NEWS/janus • Throughput under conflicts • Latency under conflicts • Overhead

Evaluation https: //github. com/NYU-NEWS/janus • Throughput under conflicts • Latency under conflicts • Overhead when there are no conflicts? • Baselines • 2 PL (2 PC) layered on top of Multi. Paxos • TAPIR [SOSP’ 15] • Testbed: EC 2 (Oregon, Ireland, Seoul)

Janus Commits under Conflicts for High Throughput-(new-order/s) 10000 Janus 1000 No aborts 2 PL

Janus Commits under Conflicts for High Throughput-(new-order/s) 10000 Janus 1000 No aborts 2 PL 100 10 Tapir 1 1 100 # Clients Aborts due to conflicts at shards & replicas TPC-C with 6 shards, 3 -way geo-replicated (9 total servers), 1 warehouse per shard.

Janus Commits under Conflicts for Low Latency 90 -percentile latency (ms) 1000 800 2

Janus Commits under Conflicts for Low Latency 90 -percentile latency (ms) 1000 800 2 PL 2 wide-area 600 roundtrips 1 wide-area roundtrip High latency due to retries after aborts Tapir 400 Janus 200 0 1 100 # Clients 2 wide-area roundtrips plus execution time TPC-C with 6 shards, 3 -way geo-replicated (9 total servers), 1 warehouse per shard.

Small Throughput Overhead under Few Conflicts 13% overhead from tracking dependencies Throughput-(txn/s) 120000 100000

Small Throughput Overhead under Few Conflicts 13% overhead from tracking dependencies Throughput-(txn/s) 120000 100000 Overhead from accept phase + increased dependency tracking Janus 80000 Tapir 60000 40000 20000 Overhead from retries after aborts 0 0. 4 0. 5 0. 6 0. 7 0. 8 Zipf coefficient 0. 9 1 Microbenchmark with 3 shards, 3 -way replicated in a single data center (9 total servers).

Related Work Isolation Level 1 RTT Commit under Conflicts Janus [OSDI’ 16] Strict-Serial ✔

Related Work Isolation Level 1 RTT Commit under Conflicts Janus [OSDI’ 16] Strict-Serial ✔ ✔ Tapir [SOSP’ 15] Strict-Serial ✔ ✖ Rep. Commit [VLDB’ 13] Strict-Serial ✔ ✖ Calvin [SIGMOD’ 12] ✖ ✔ Spanner [OSDI’ 12] MDCC [Euro. Sys’ 13] Strict-Serial EPaxos [SOSP’ 13] Strict-Serial ✖ Rococo [OSDI’ 14] Read. Commit* ✔ ✖ ✖ COPS [SOSP’ 11] Causal+ ✔ ✔ Eiger [NSDI’ 13] Causal+ ✔ ✔

Conclusion • Two limitations for layered transaction protocols • Multiple wide-area round trips in

Conclusion • Two limitations for layered transaction protocols • Multiple wide-area round trips in the best case • Conflicts cause aborts • Janus consolidates concurrency control and consensus • Ordering requirements are similar and can be combined! • Establishing a single ordering with dependency tracking enables: • Committing in 1 wide-area round trip in the best case • Committing in 2 wide-area round trips under conflicts • Evaluation • Small throughput overhead when there are no conflicts • Low latency and good throughput even with many conflicts