Using Open Stack Sahara Manila to run Analytics

MISTRAL MURANO TROVE KOLLA MAGNUM Open. Stack Keeps Growing! CONGRESS IRONIC CEILOMETER ZAQAR Open.

Lots of data in shared file systems NFS CIFS. Gluster _ HDFS. … 3

Open. Stack Foundation User Survey Results Shared File System Drivers in Use (April 2016)

Open. Stack Manila The Open. Stack Shared File Service Program § Manila delivers file

Hadoop Clusters are not easy! Same for Spark, Storm, etc… § Difficult to configure

Open. Stack Sahara The Open. Stack Data Processing Program § Put Hadoop and alike

Technical Overview Virtual Cluster Computing Task Manila Service HDFS Driver VM HDFS Host Net.

Summary § Manila adds Fileshares-aa. S to Open. Stack § Sahara adds Analytics-aa. S

Slides: 9

Download presentation

MISTRAL MURANO TROVE KOLLA MAGNUM Open. Stack Keeps Growing! CONGRESS IRONIC CEILOMETER ZAQAR Open. Stack Today HORIZON BARBICAN KARBOR SAHARA NOVA SWIFT SENLIN CINDER HEAT NEUTRON MANILA GLANCE KEYSTONE TACKER TRIPLE-O DESIGNATE MONASCA HORIZON KURYR CHARMS RALLY

Open. Stack Foundation User Survey Results Shared File System Drivers in Use (April 2016) [CATEGORY [CATEGO NAME] RY NAME][PERCENTA [CATEGORY [PERCENT GE] NAME] AGE] [PERCENTA GE] HDFS 8% Windows SMB 12% [CATEGORY NAME] [PERCENTAG EMC 5% E] Quobyte 3% Huawei 3% § Net. App is the founder of, leading contributor to, and most widely deployed option for Manila 4 © 2017 Net. App, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Open. Stack Manila The Open. Stack Shared File Service Program § Manila delivers file systems as a service: § NFS, CIFS, HDFS, and more Manila ONTAP § Production-ready in Liberty § Supports a wide range of network topologies 5 © 2017 Net. App, Inc. All rights reserved. — NETAPP CONFIDENTIAL — Open. Stack Compute Admin R&D Eng VM VM

Hadoop Clusters are not easy! Same for Spark, Storm, etc… § Difficult to configure & maintain § Adaption to changing demands § Hardware headaches § Commodity hardware is cheap but requires more work § Reliable hardware is expensive § Hadoop dev is very difficult without a real cluster 6 © 2017 Net. App, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Open. Stack Sahara The Open. Stack Data Processing Program § Put Hadoop and alike in a Open. Stack cloud! § Then have easy-to-use, standardized interfaces: § § § To create clusters (reliably and repeatedly) To scale clusters To run data processing jobs On any popular data processing framework With sensible defaults that just work And sophisticated configuration management for expert users § That's Open. Stack Sahara! 7 © 2017 Net. App, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Technical Overview Virtual Cluster Computing Task Manila Service HDFS Driver VM HDFS Host Net. App Hadoop NFS Connector VM Manila Service NFS Driver NFS Host Local Volume VM Manila Service Gluster Drivers or others Gluster. FS Host External HDFS via Manila NFS directly via Manila Local Storage in Manila § Sahara can support external HDFS by using the HDFS driver in Manila § Net. App Hadoop NFS Connector brings the NFS capability into Hadoop § Use local storage in Hadoop to remote mount any type of storage in Manila 8 © 2017 Net. App, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Summary § Manila adds Fileshares-aa. S to Open. Stack § Sahara adds Analytics-aa. S to Open. Stack § Net. App NFS Connector for HDFS allows access of NFS data through HDFS protocol § All together: Easy analytics against existing datasets in file shares § Resources § https: //www. youtube. com/watch? v=Tf_8 v 0 Ov. R 80 § https: //github. com/Net. App-Hadoop-NFS-Connector § https: //www. netapp. com/us/media/tr-4464. pdf 9 © 2017 Net. App, Inc. All rights reserved. — NETAPP CONFIDENTIAL —