Introducing SAS Grid Manager for Hadoop Cheryl Doninger

Introducing – SAS® Grid Manager for Hadoop Cheryl Doninger, SAS Doug Haigh, SAS

About the presenter I started with SAS in 1986 and am currently a Senior R&D Director. My teams work on many of the foundation technologies providing the compute capabilities of SAS: SAS Grid, SAS/CONNECT, all host teams, Core, IOM and Work. Space as well as SAS Environment Manager. I have a Master’s from NC State and hold 2 patents related to SAS Grid Computing.

A Bit of Background… § SAS Grid designed for § Workload management § High availability § Performance § Architected to support multiple providers § Platform Suite for SAS (LSF, PM) § Hadoop (YARN, Oozie)

Why SAS Grid Manager for Hadoop? § Co-location of SAS Grid jobs on Hadoop cluster § Requires – § § § Integration with YARN Supported enterprise Hadoop distribution Kerberos Spare capacity to accommodate additional workload Nodes architected to be compute nodes

What Behavior Can I Expect? § Consistent job submission § § Grid launched WS servers Grid servers created with SAS/CONNECT Batch submission with SASGSUB Batch submission with Schedule Manager plug-in § All SAS Grid integration is the same § SAS Grid jobs will run unchanged

What Is Different? § Hadoop is not part of the product § Monitoring/management via Hadoop interfaces § Kerberos is required § Need for shared file system is reduced § Data will need to be migrated to HDFS § Need to understand the capabilities of YARN § Each SAS Grid job results in (at least) two YARN containers

Conclusion § Gas powered vs. electric