JES Application Scheduler Technical Overview 152022 1 JESAS

JES- Application Scheduler Technical Overview 1/5/2022 1

JES-AS Key Motivators Need to comprehensive and portable solution in meeting current data scheduling needs Need to have interfaces with different transport(Interaction) technologies Current day schedulers choke, with one large computation job, which gobbles up computation power and time. This creates havoc situation for periodic jobs. As The time slot for batch applications is limited and fixed Designed for Scalability and Performance for large computational intensive jobs. Computation Unit needs to be additive in terms of adding software/hardware instance Need to have useful management/monitoring consoles to easy the operational effort for IT personal Minimal effort for developers for writing large complex batch processing tasks. 1/5/2022 2

JES-AS Domain Elements Job: A batch job is a program that is assigned to the computer to run without further user interaction Job Scheduling: By definition, job scheduling concerns executing the right job, on the right day, at the right time, after the right dependency Calendar: Plan for executing jobs based on calendar events Type of Jobs : Periodic jobs, Daemons, Event Triggered Jobs Tasks : ANT Tasks, FTP Tasks, Java Tasks, Executables, RMI tasks, Web Services Tasks, Globus Tasks etc. Group: Group of related Jobs 1/5/2022 3

JES-AS Schematic View RMI, Corba, TCP, UDP, Local, MQ Webservices, SOAP Online Applications RMI, Corba, TCP, UDP, Local, MQ Webservices, SOAP Job Event Queue ASync Event Queue Job Executor Agent Node 1 Scheduling Engine Job Executor Agent Job Event Queue Management/ Monitoring Console Job Repository Node 2 Java/J 2 EE Platform (JVM, J 2 EE. Servlet, MDB, Ant, FTP, Globus) 1/5/2022 4

JES-AS Scheduling Engine Stack Crypto & Security Utils Service Broadcaster RMI Corba Web Services MQ Scheduling Engine Agent Discovery & Enlistment Manager Plug gable Scheduling Strategy Local Error Async Event Receiver Event Queue Proxy Layer Job Repository Event Transport Proxy Layer RMI Corba Job Re-Submission Queue UDP/TCP Job Executor Agents Web Services MQ UDP/TCP Job Re. Submission Manager Local Job Status Queue JMX MBeans 1/5/2022 5

JES-AS Job Executor Agent Stack SYNCPOINT – Tracker For Recovery Utils RMI Corba Web Services Job Executor MQ UDP/TCP Local Event Queue Event Receiver Proxy Layer Service Broadcaster Globus SOAP Ant Executables EJB JUNIT JDBC LDAP JVM FILE Web Services Others Task Adapters Crypto & Security Utils JMX MBeans 1/5/2022 6

JES-AS Scheduler Engine Sub-Components Scheduling Engine : Responsible for controlling the execution of the jobs based on Job Dependencies and Calendars. This component is heart of all control activities and is responsible for the run-time behavior of the component. Typically would be a Controller class for the component Pluggable Scheduling strategy Responsible for scheduling jobs based on calendar, inter-job and group dependences and execution status of the jobs. This component should be loosely coupled for providing users with an option of plugging custom scheduling strategy. 1/5/2022 7

JES-AS Scheduler Engine Sub-Components Event Receiver Proxy Layer : This layer acts as an interface layer for Online/EIS systems to dynamically post Job events. The responsibility of this layer is provide transport technology abstraction to underlying scheduling engine. Job Repository: Job Repository contains all information in-terms of Job Definition, Control Definitions, etc. The information might be stored in XML files, Database or LDAP or any other persistence mechanism. The Responsibility of this component is abstract underlying persistence technology and also provide basic data model for the component. 1/5/2022 8

JES-AS Scheduler Engine Sub-Components Job Resubmission Manager : The responsibility of this sub-component is to re-submit the jobs, incase all types of failures (Failure in Execution, Failure in JOB Executor Agent , Network etc. ) Job Repository: Job Repository contains all information in-terms of Job Definition, Control Definitions, etc. The information might be stored in XML files, Database or LDAP or any other persistence mechanism. The Responsibility of this component is abstract underlying persistence technology and also provide basic data model for the component. 1/5/2022 9

JES-AS Scheduler Engine Sub-Components Event Transport Proxy Layer : This layer acts as an interface layer for Scheduling Engine and Job Executor Agents. The responsibility of this layer is provide transport technology abstraction to underlying scheduling engine and transporting job execution requests to underlying agents MBeans – JMX component This layer provides Mbeans, Listeners and counters for management and monitoring. Management Consoles are usually built using this layer. 1/5/2022 10

JES-AS Scheduler Engine Sub-Components Broad. Caster The responsibility of this layer to broadcast using standard technologies SNMP, UDP about services ports for Job Executor Agents to communicate with Scheduler Engine Agent Enlistment Manager The responsibility of this layer is act on behalf of the Job Executor Agents for purpose of enlistment for accepting job requests for Scheduler Engine. Initial, the focus would be to provide static enlistment. In Static Enlistment, the agent address and other relevant parameters necessary for Interaction, shall be available in configuration file of Enlistment Manager. In Future, the enlistment can be a dynamic process, where Job Executor Agents enlist them self using standard protocols 1/5/2022 11

JES-AS Scheduler Engine Sub-Components Crypto and Security Utilities These utilities provide basic infrastructure for mutual authentication of Scheduler Engine and Agents. The Crypto Utilities help in encrypting the data when passed over the wire. 1/5/2022 12

JES-AS Job Executor Agents Sub-Components Event Receiver Proxy Layer It is an interface layer between Scheduler Engine and Job Executor Agent/ The responsibility of this layer is provide transport technology abstraction and deliver Job Requests to underlying Job Executor The workhorse, responsible for executing the job in sequential or parallel manner (based information provided in Job definition) with the help of underlying Task Adapters 1/5/2022 13

JES-AS Job Executor Agents Sub-Components Task Adapters Layer Tasks are kind of ready made-reusable functions, which are often used. The tasks are quite analogous to ANT tasks. Tasks help users to concentrate on business logic or involve very minimal or no-coding for mundane kind of jobs. For Example – – A JOB defined for using Ant Task, shall invoke a ant build file specified in JOB definition file – A Job Defined to invoke a web service, shall invoke a web service specified in JOB definition file. 1/5/2022 14

JES-AS Job Executor Agents Sub-Components Crypto and Security Utilities These utilities provide basic infrastructure for mutual authentication of Scheduler Engine and Agents. The Crypto Utilities help in encrypting the data when passed over the wire. Sync Point Utilities Help in Jobs to keep track of execution from transaction perspective. This sync point should help in restarting the jobs from point last exited. 1/5/2022 15

Q&A 1/5/2022 16