SAS Grid 101 How It Can Modernize Your

SAS® Grid 101: How It Can Modernize Your Existing SAS Environment Cheryl Doninger R&D Director, SAS Manager, SAS Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Glenn Horton Software

The Need to Modernize. . . Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Enterprise IT Challenges Increased data growth Higher availability flexibility to maintain and upgrade Frequent server crashes Many users on single resource Need to grow incrementally Predict long term capacity and budget planning Need reduced complexity SMP increased obsolete or manageability at capacity Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Enterprise Application Needs Increased data growth Production jobs w/ complex schedules or dependencies Parallel production jobs Interactive SAS sessions Many users on single resource Short ad-hoc interactive jobs Longer ad-hoc command line batch jobs Interactive EG users Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS Grid Manager Distributed Enterprise Scheduling Distribute jobs within workflows to a range of hosts. Multi-User Workload Balancing Distribute workloads to a shared pool of resources. Parallelized Workload Balancing Distribute parallelized SAS workloads to a shared pool of resources. Optimize the Efficiency and Utilization of Computing Resources Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Three Paths to Grid Enabling SAS Products and Solutions Automatic Parallel Code Generation SAS Program with parallel sub-tasks Add Grid Computing Program Header Create Sub-Tasks Using 4 GL Syntax SAS Programs Add Grid Computing Program Header Create Workflow and Schedule It Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Storage Performance is Critical in a Grid Mounted Volume 1 Grid Node 1 Fiber Grid Node n Channel Switch Shared File System Volume 1 Shared Storage Device(s) Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS DI Studio – Enterprise Scheduling Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS DI Studio – Multi-User Workload Balancing Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS DI Studio – Parallelized Workload Balancing Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS Enterprise Miner – Parallelized Workload Balancing Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Phased Approach Phase 1 risk e z i m mini ange h c mize i n i m Phase 2 Phase 3 Multi-User Jobs on the Grid Schedule Production Jobs Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Migrate Parallel Workloads

Workload Balancing - Interactive § Save following to external file (c: gpre. sas for example) options noconnectpersist; options noconnectwait; options metaserver='dnnnn'; options metaport=8561; %let rc=%sysfunc(grdsvc_enable(grid, resource=SASMain)); signon grid; rsubmit; § Create new DMS key definition (F 12 for example) gsubmit “%include ‘c: gpre. sas’; ”; rsubmit; Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Workload Balancing - Batch § Download Unix script file from Scalability site. /runsas. sh <sas file_name> <extra SAS options> § Where runsas. sh will do the following: • • • create a temporary file write necessary grid and SAS/Connect syntax to file copy contents <sas file_name> to temporary file submit contents of file and <extra SAS options> to the grid perform error checking and messaging in event of errors Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Deploy a SAS Program for Scheduling Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Create a New Flow Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Schedule the Flow Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Parallelized Workload Balancing %let rc=%sysfunc(grdsvc_enable(_all_, resource=SASMain)); signon task 1; rsubmit task 1 wait=no; /* code to be remote submitted */ endrsubmit; signon task 2; rsubmit task 2 wait=no; /* code to be remote submitted */ endrsubmit; waitfor _all_ task 1 task 2; /* continue local execution */ Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS Grid Application - Results § The application • • Proc IML calculate Pairwise test statistic compare 2 matrices, 4600 columns, 40 rows 21, 000 total comparisons § The results • serial execution 16: 16 • parallel grid execution over 5 cpus 2: 38 • nearly linear scalability Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Setting Name of Grid Jobs § Job names can be set in SAS code %let jnvar=Finance. Summary; %let jovar=queue=priority; %put %sysfunc(grdsvc_enable(_ALL_, server=SASApp; jobname=jnvar; jobopts=jovar)); signon t; 20 Better Job Control Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Setting Name of Grid Jobs Results 21 Better Job Control Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Setting Grid Job Options § Job options can be set in SAS code %let jnvar=Finance. Summary; %let jovar=queue=priority; %put %sysfunc(grdsvc_enable(_ALL_, server=SASApp; jobname=jnvar; jobopts=jovar)); signon t; 22 Better Job Control Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Setting Grid Job Options § Job options can be set in SASMC 23 Better Job Control Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Setting Grid Job Options - Results 24 Better Job Control Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Queue Definitions Begin Queue QUEUE_NAME = priority PRIORITY = 40 DESCRIPTION = high priority users End Queue Begin Queue QUEUE_NAME = normal Begin Queue QUEUE_NAME = night PRIORITY = 30 DISPATCH_WINDOW = (18: 00 -07: 30) RUN_WINDOW = (18: 00– 08: 00) HOSTS = all ~host 1 DESCRIPTION = night time batch jobs End Queue PRIORITY = 30 Begin Queue PREEMPTION = PREEMPTABLE[short] QUEUE_NAME = short DESCRIPTION = default queue PRIORITY = 35 PREEMPTION = PREEMPTIVE[normal] End Queue DESCRIPTION = for jobs of short duration End Queue Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

SAS Grid Manager – Load Balancing High Priority Queue HOST A Normal Queue SAS Application SAS Grid Manager HOST B Night Queue Short Queue HOST C Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Summary § § § § Centralized, shared environment Dynamic resource allocation to meet peak demand Policies and prioritization for use of resources Run large more complex analysis De-couple applications from infrastructure Ease maintenance of computing infrastructure Improve price/performance with commodity hardware Scale out cost effectively as needs grow Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Want to Know More? § Introducing the SAS ® Code Analyzer, Mon, 3: 00 214 A, Paper 006 -2008 § Using SAS/OR® and SAS® Grid Manager to Solve Optimization Problems on the Grid, Mon, 5: 00 209, Paper 203 -2008 § Architecting a Finely Tuned SAS® Grid Solution, Tues, 8: 00 214 B, Paper 390 -2008 § Balancing the Load: SAS® Server Technologies for Scalability, Tues, 8: 30 214 B, Paper 391 -2008 § Data Integration in a Grid-Enabled Environment, Wed, 8: 00 217 B, Paper 133 -2008 § SAS® Grid 101: How It Can Modernize Your Existing SAS Environment, Wed, 11: 00 Ballroom C 1, Paper 046 -2008 Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Copyright © 2007, SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.