Apache Zoo Keeper MAHADEV KONAR What is Zoo

Apache Zoo. Keeper MAHADEV KONAR

What is Zoo. Keeper? A highly available, scalable, distributed coordination kernel

Use Cases » Leader Election » Group Membership » Work Queues » Event Notifications/workflow management » Configuration Management » Cluster Management » Sharding

What is Zoo. Keeper again? � File api without partial reads/writes � No renames � Ordered updates and strong persistence guarantees � Conditional updates (version) � Watches for data changes � Ephemeral znodes � Generated file names

Data Model � Hierarchal namespace � Each znode has data and / apps app 1 children � data is read and written in its entirety servers regionserver master locks read-1 users

Zoo. Keeper API String create(path, data, acl, flags) void delete(path, expected. Version) Stat set. Data(path, data, expected. Version) (data, Stat) get. Data(path, watch) Stat exists(path, watch) String[] get. Children(path, watch)

Zoo. Keeper Service Leader Server Client Server Client � All servers store a copy of the data (in memory) � A leader is elected at startup � Followers service clients, all updates go through leader � Update responses are sent when a majority of servers have persisted the change

Zoo. Keeper and HBase �Master Failover �Region Servers and Master discovery via Zoo. Keeper HBase clients connect to Zoo. Keeper to find configuration data Region Servers and Master failure detecti 0 n

Hbase and Zoo. Keeper as of now! shutdown / root-region-server rs master • Master • If more than one master, they fight • Root Region Server • This znode holds the location of the server hosting the root of all tables in hbase • rs • A directory in which there is a znode per Hbase region server • Region Servers register themselves with Zoo. Keeper when they come online • On Region Server failure (detected via ephemeral znodes and notification via Zoo. Keeper), the master splits the edits out per region

Common Problems/Error Cases �Garbage Collection at the Region Servers � Causes zookeeper clients to stall Session expiry �Low throughput and connection loss Mostly due to under provisioned Zoo. Keeper instances Disk and Memory usage �Bad Usage example: Name. Node, Region. Server, Job. Tracker, Zoo. Keeper running on the same node

Release 3. 3. 0, whats in for Hbase? �Allow configuration of session timeout min/max bounds � HBase needs large session timeouts �Improved logging information to detect issues �Improved debugging tools �Improved documentation �Improved performance and robustness �Queue implementation available

Upcoming 3. 4 release �No Connectionloss �Use Netty - allow encryption �Testing Mockito �More of backwards compatibility testing

More Zoo. Keeper in Hbase? �Table Schema and state in Zoo. Keeper read only, online �Region Server state transitions via Zoo. Keeper �Store region assignment in Zoo. Keeper for each Region Server �http: //wiki. apache. org/hadoop/Zoo. Keeper/HBase. U se. Cases

Questions?