APACHE SLING FRIENDS TECH MEETUP BERLIN 22 24

APACHE SLING & FRIENDS TECH MEETUP BERLIN, 22 -24 SEPTEMBER 2014 Oak, the Architecture of the new Repository Michael Dürig, Adobe Research

Design goals § § § adapt. To() 2014 Scalable Big repositories Clustering Customisable, flexible OSGi friendly 2

Outline § CRUD § Changes § Search adapt. To() 2014 3

Tree model a b adapt. To() 2014 d c 4

Updating ? a b adapt. To() 2014 d c x 5

MVCC HEAD r 1: / a r 2: / d r 1: /d b r 1: /a/b r 2: /a/b adapt. To() 2014 r 2: /d c r 2: /d/x 6

Refresh and Garbage Collection adapt. To() 2014 7

Refresh garbage adapt. To() 2014 8

Garbage collection garbage adapt. To() 2014 9

Concurrency and Conflicts adapt. To() 2014 10

Concurrent updates r 2 a adapt. To() 2014 r 1 r 2 b 11

Merging r 2 a merge upates r 1 r 3 r 2 b adapt. To() 2014 12

Conflict handling: serialisation § Fully serialised § Fail, no concurrent update § Partially serialised § Concurrent conflict free updates adapt. To() 2014 13

Conflict handling strategies: merging § Partial merge § Conflict markers, deferred resolution § Full merge § Need to choose victim adapt. To() 2014 14

Replicas and Sharding adapt. To() 2014 15

Replica and caches master copy adapt. To() 2014 full replica cache 16

Sharding strategies by path adapt. To() 2014 by level by hash with caching 17

Implementations adapt. To() 2014 18

Micro. Kernel / Node. Store § Tree / Revision model implementation Responsible for Not responsible for Clustering Validation Sharding Access control Caching Search Conflict handling Versioning adapt. To() 2014 19

Current implementations Document. MK Tar. MK (Segment. MK) Persistence Mongo. DB, JDBC Local FS Conflict handling Partial serialisation Full serialisation Clustering Mongo. DB clustering Simple failover Sharding Mongo. DB sharding N/A Node Performance Moderate High Key use cases Large deployments (>1 TB), Small/medium concurrent writes deployments, mostly read adapt. To() 2014 20

Access Control adapt. To() 2014 21

Accessible paths a b adapt. To() 2014 d c 22

Existentialism § All paths traversable § Node may not exist § Decorator on Node. Store root. get. Child. Node("a"). exists(); � false root. get. Child. Node("a"). get. Child. Node("b"). exists(); � true adapt. To() 2014 23

Comparing Revisions adapt. To() 2014 24

Content diff § What changed between trees § Cornerstone for § § adapt. To() 2014 Validation Indexing Observation … 25

What changed? ∆ adapt. To() 2014 26

Example: merging r 2 a r 1 r 2 b adapt. To() 2014 ∆ r 1 ➞ r 2 a “a” modified “b” removed r 3 ∆ r 1 ➞ r 2 b “d” modified “x” added 27

Commit Hooks adapt. To() 2014 28

Commit hooks § Key plugin mechanism § Higher level functionality § § § adapt. To() 2014 Validation (node type, access control, …) Trigger (auto create, defaults, …) Updates (index, …) 29

Editing a commit ∆ adapt. To() 2014 ∆+x 30

Commit hooks § Based on content diff § pass a commit § fail a commit § edit a commit § Applied in sequence adapt. To() 2014 31

Type of hooks Commit. Hook Editor Validator Content diff Optional Always Can modify Yes No Programming model Simple Callbacks Performance impact High Medium Low adapt. To() 2014 32

Observers adapt. To() 2014 33

Observers § Observe changes § § After commit Often does a content diff Asynchronous Optionally synchronous § adapt. To() 2014 Local cluster node only 34