Architecture and Interface of Scalable Distributed Database System
Architecture and Interface of Scalable Distributed Database System SD-SQL Server Soror SAHRI Soror. sahri@dauphine. fr Witold LITWIN Witold. litwin@dauphine. fr Ceria Laboratory Paris-Dauphine University Thomas SCHWARTZ tjschwarz@scu. edu Comp. Eng. Dep. Santa Clara U. The IASTED International Conference on DBA 2006 IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006
Outline w Introduction w Overall Architecture w Application Interface w Implementation w Conclusion IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 2
Introduction w Most of DBSs have distributed/parallel versions n SQL Server, Oracle, DB 2 w DBSs do not provide dynamically scalable tables. n All require manual repartitioning when tables scale-up. Solution? A Scalable Distributed Database System: SD-DBS IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 3
Introduction Applies SDDS technology to DBSs Why SDDSs? üProvide many scalable distributed partitioning schemes. üLH*, RP*, k-RP*, LH*RS… ü These schemes can serve as the basis for SD-DBS architecture IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 4
SD-SQL Server Architecture User/Application sd_insert sd_select SD-SQL Server Managers Linked SQL Servers SD-SQL client SD-SQL server SD-SQL peer D 1_T T _D 1_T NDBs SD-SQL server D 1 IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 D 2 Split _D 1_T Di+1 5
SD-SQL Server Architecture Nodes, SDBs, NDBs MDB Node 1 DB 1 Node 2 DB 1 SDB IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 Node 3 DB 1 DB 2 …… Node i DB 1 DB 2 SDB 6
SD-SQL Server Architecture Scalable Tables: Segments w A scalable (distributed) table is a collection of segments n Segments are SQL tables w A scalable table has, initially, only one primary segment n At some server or peer NDB w The number of segments in a scalable table is variable. n If a segment overflows, its split is triggered IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 7
SD-SQL Server Architecture Scalable Tables: Split w A split occurs when an insert overflows the segment capacity w Splits produce other segments for a scalable table. n Each is located at a different NDB l Within the SDB w If there is not enough NDBs, splits dynamically append new ones IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 8
SD-SQL Server Architecture Images hide the scalable table segments u An image is a distributed updateable partitioned view of a scalable table u u u An image presents the scalable table partitioning u u Union-all view with check constraints It do not address any new segments resulted from a split Each scalable table has only one primary image and one or several secondary images IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 9
SD-SQL Server Architecture Images DB 1 SDB N 1. DB 1 Primary Image N 2. DB 1 … Ni. DB 1 T Scalable Table CREATE VIEW T AS SELECT * FROM N 1. DB 1. _N 3_T UNION ALL SELECT * FROM N 2. DB 1. _N 3_T UNION ALL SELECT * FROM Ni. DB 1. _N 3_T IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 10
SD-SQL Server Application Interface w The application interface manipulates scalable tables through SD-SQL Server commands. w The SD-SQL Server commands start with ‘sd_’ to distinguish from SQL Server commands for static tables. INSERT CREATE TABLE IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 sd_insert sd_create_table 11
SD-SQL Server Application Interface w Use of the Sky. Server DB as benchmark http: //research. microsoft. com/~gray/SDSS n Photo. Obj table as a scalable table. n Photo. Obj has 158, 426 tuples (about 260 MB) w Use of the laboratory machines n Ceria, Dell 1, Dell 2…. IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 12
SD-SQL Server Application Interface Nodes Management w Node Creation n sd_create_node ‘Dell 1’ /* Server by default */ n sd_create_node ‘Ceria’, ‘client’ w Node Alteration n sd_alter_node ‘Ceria’, ‘ADD server’ /* Becomes peer*/ w Node Removal n sd_drop_node ‘Ceria’ IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 13
SD-SQL Server Application Interface SDB/NDB Management w SDB Creation n sd_create_scalable_database ‘Sky. Server’, ‘Dell 1’, ‘Server’, 2 /* Creates the primary Sky. Server NDB as well at Dell 1*/ w SDB Alteration n sd_create_node_database ‘Sky. Server’, ‘Ceria’, ‘Client’ w SDB Removal n sd_drop_scalable_database ‘Sky. Server’ IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 14
SD-SQL Server Application Interface Scalable Tables w Scalable Table Creation n sd_create_table ‘Photo. Obj (objid BIGINT PRIMARY KEY…)’, 10000 w Scalable Table Alteration n sd_alter_table ‘Photo. Obj ADD t INT’, 1000 sd_create_index ‘run_index ON Photoobj (run)’ sd_drop_index ‘Photo. Obj. run_index’ w Scalable Table Removal n sd_drop_table ‘Photo. Obj’ IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 15
SD-SQL Server Application Interface Images w Secondary Image Creation n sd_create_image ‘Ceria’, ‘Photo. Obj’ n sd_create_image ‘Ceria 2’, ‘Photo. Obj’ w Secondary Image Removal n sd_drop_image 'Photo. Obj’ IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 16
SD-SQL Server Application Interface Scalable Queries USE Skyserver /* SQL Server command */ w Scalable Update Queries n sd_insert ‘INTO Photo. Obj SELECT * FROM Ceria 5. Skyserver-S. Photo. Obj’ w Scalable Search Queries n sd_select ‘* FROM Photo. Obj’ n sd_select ‘TOP 5000 * INTO Photo. Obj 1 FROM Photo. Obj’, 500 IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 17
SD-SQL Application Interface IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 18
SD-SQL Application Interface Command Processing w SD-SQL Server uses distributed stored procedures with dynamic SQL… w SD-SQL Server processes every command as a distributed transaction at Repeatable Read isolation level w See details in the papers n n “SD-SQL Server: a Scalable Distributed Database System”, CERIA Research Report 2005 -12 -13, December 2005 “Overview of Scalable Distributed Database System SD-SQL Server”, Intl. Workshop on Distributed Data and Structures, WDAS 2006, Santa Clara, CA, Carleton Scientific. IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 19
Conclusion w Scalable distributed databases with scalable tables are now a reality with SD-SQL Server n No more manual repartitioning l Unlike in any other DBS we know about w See the “Related Work” in the paper w The performance analysis proves n n Efficiency of our design Immediate utility of SD-SQL Server IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006 20
For more details: http: //ceria. dauphine. fr Thank you. IASTED-DBA 2006, Innsbruck, Feb 15 th, 2006
- Slides: 21