Firewater The split personality of Lucid DB A

  • Slides: 10
Download presentation
Firewater The “split” personality of Lucid. DB A quick whirl through combining a purpose

Firewater The “split” personality of Lucid. DB A quick whirl through combining a purpose built BI database with scale “out” capabilities aka – Lucid. DB roadmap for “Bigger and Faster”

 First and only 100% Open Source DB purpose built for doing Analytics Long

First and only 100% Open Source DB purpose built for doing Analytics Long Code History (10+ years) Foundation backing Commercial Support through Dynamo. BI

 Column Store Vertical Partitioning (solves I/O problems) Great for BI, awful for OLTP

Column Store Vertical Partitioning (solves I/O problems) Great for BI, awful for OLTP Bitmap IDXes, Hash Joins/Aggs Labels (read consistent reporting) UDXs, plugin framework, extendable catalog

We're winning the “I/O war” Row Store Column Store I/O Bound I/O efficient CPU

We're winning the “I/O war” Row Store Column Store I/O Bound I/O efficient CPU Cycles to Spare CPU bound GETTING AHEAD OF THE I/O CURVE GIVES US A NEW WAR: CPU and Massively Parallel Processing THOU SHALT PARALLELIZE THY PROCESSING THY DATA SHALL BE PARTITIONED

SELECT SUM(SALES) FROM 100 M_ROW_TBL GROUP BY YEAR, MONTH SERVER 1 LUCIDDB SELECT SUM(SALES)

SELECT SUM(SALES) FROM 100 M_ROW_TBL GROUP BY YEAR, MONTH SERVER 1 LUCIDDB SELECT SUM(SALES) FROM 100 M_ROW_TBL GROUP BY YEAR, MONTH

 1 SERVER, SMP, 8 CPUs Available First “run” thrown out to tickle the

1 SERVER, SMP, 8 CPUs Available First “run” thrown out to tickle the cache SELECT T. "Carrier" as "CARRIER", C, C 2, C*1000/C 2 as "C 3" FROM (SELECT "Carrier", count("Carrier") AS C FROM OTP. "ontime" WHERE "Dep. Delay">10 GROUP BY "Carrier") T JOIN (SELECT "Carrier", count("Carrier") AS C 2 FROM OTP. "ontime" GROUP BY "Carrier") T 2 ON (T. "Carrier" = T 2. "Carrier") ORDER BY 3 DESC;

DEMO

DEMO

SMP MPP

SMP MPP

What's Working Projection Pushdown Filter Pushdown ( where j > 3) GROUP BY pushdown

What's Working Projection Pushdown Filter Pushdown ( where j > 3) GROUP BY pushdown (all aggs) GROUP BY + distinct count Combinations of these things

What's Not (aka Roadmap) Hash and Replicated tables and * schema Joins Parallel Loading

What's Not (aka Roadmap) Hash and Replicated tables and * schema Joins Parallel Loading GOAL: Full support for Mondrian * queries Lucid. DB Firewater * queries MONDRIAN Page Views Cube