Open Frame Introduction Batch Tmax Soft Co Ltd
Open. Frame Introduction Ⅱ (Batch) Tmax. Soft Co. , Ltd © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 0/40
Contents 1. Prerequisite Knowledge 1. 1 MVS JCL 1. 2 TP-monitor Tmax 4. TSO/CLIST 4. 1 TSO/E & CLIST 5. Features 2. Open. Frame Batch 2. 1 Open. Frame Batch Introduction 2. 2 Open. Frame Batch Components 2. 3 Open. Frame Batch Key Features 3. Job Management 3. 1 Introduction 3. 2 JOB Flow 3. 3 JOB Scheduling 3. 4 JOB Group 3. 5 JOB Life Cycle 3. 6 Internal Reader 3. 7 Runner 3. 8 TEXTRUN 3. 9 TJES Servers 3. 10 SPOOL © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 5. 1 Security Features 5. 2 Multi-Node Environment 5. 3 Failover Q&A
1. Prerequisite Knowledge 1. 1 MVS JCL 1. 2 TP-monitor Tmax © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved.
1. 1 MVS JCL (Job Control Language) specifies the following: • Who you are (identity) • Resources (program, dataset, class, etc. ) needed to process a JOB • Order of programs to be executed • Condition of failure //SORTJOB CLASS=A, MSGCLASS=X, MSGLEVEL=(1, 1), // COND=((12, EQ), (16, EQ)), // USER=ROOT, PASSWORD=**** //RMSTEP EXEC PGM=IEFBR 14 //RMDD 01 DD DSN=SORT. OUT 01, DISP=(MOD, DELETE) //SORT 01 EXEC SORT //SORTIN DD DSN=SORT. IN 01, DISP=SHR //SORTOUT DD DSN=SORT. OUT 01, DISP=(NEW, CATLG, DELETE), RECFM=FB, LRECL=80 //SYSOUT DD SYSOUT=* //SYSIN DD * SORT FIELDS=(1, 1, CH, A) */ © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 3/40
1. 2 TP-monitor Tmax TP-monitor Server Platform Transparency Network Transparency Application and Tool Support(APIs) RDBMS Support Single System Login Enhanced Security Location Transparency Heterogeneous Join, Replication Transaction Monitoring, Queuing Process Management Client © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 4/40
1. 2 TP-monitor Tmax • Distributed Transaction Processing Platform Open. Frame • Fully compliant with X/Open DTP standards • Variety of system level services Open. Frame Batch Based on Tmax • Muti-node Environment Tmax • System Clustering • Load Balancing • Failover • Connectivity with Legacy Systems © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 5/40
2. Open. Frame Batch 2. 1 Open. Frame Batch Introduction 2. 2 Open. Frame Batch Components 2. 3 Open. Frame Batch Key Features © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved.
2. 1 Open. Frame Batch Introduction Open. Frame Batch Architecture Applications Open. Studio Open. Frame/Batch TJES Runner/Spool/Output Third Party Products & Utilities TSO/E System Utilities Tmax Security TACF RDBMS Open. Frame TSAM Job Scheduler Open. Frame Hi. DB Open. Frame NVSM UNIX * Webto. B (Web Server), JEUS (J 2 EE App Server), RCA (Tmax Raw Client Agent) • Open. Frame Batch runs on top of Tmax TP-monitor to leverage Tmax’s load-balancing and failover features. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 7/40
2. 1 Open. Frame Batch Introduction Open. Frame Batch Process USER ACTIONS Define the JOB Create the JCL SYSTEM ACTIONS Submit the JCL TJES interprets JCL & passes it to Runner executes the JOB System Messages TJES collects output & info about the JOB User views output • Open. Frame Batch consists of various components that simulate a mainframe batch environment to run a batch JOB under open environment. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 8/40
2. 2 Open. Frame Batch Components TJES • Stands for Tmax Job Entry Subsystem • A core component of Open. Frame/Batch. Equivalent to a mainframe’s JES • Uses JOBQ/OUTQ/Spool for processing & managing JOBs • Supports output processing through a printer & internal reader (JCL submission) • Built on top of Tmax TP-monitor in order to support automated error recovery and parallel Sysplex features Runner • Open. Frame/Batch module that parses JCL statements and runs an actual JOB System Utilities • Various Batch Utilities & Third Party Utilities TSO/E • Provides execution of CLIST and interactive processing with the user at a terminal. User Interface • TJESMGR (CUI) and Open. Studio (GUI) © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 9/40
2. 3 Open. Frame Batch Key Features Open. Frame/Batch provides a mainframe JES-compatible environment and powerful batch job management functions. Job Management TSO/E Provides Diverse Batch Utilities • Job management function based on various scheduling policies. • Supports user interactive programs. • Equipped with various utilities (IDCAMS, FTP and etc. ) • Provides an integrated interface for job execution, cancellation, monitoring, and for checking results. • Transmits datasets to and from a PC. • Supports CLIST scripts. • Supports various third party utilities • Supports Ez. Plus • Supports SORT via various sort engines (Pro. Sort, Sync. Sort, Powerb. Sort) JCL Support Powerful Clustering Function Enhanced Management Function • Supports MVS/JCL, MSP/JCL, and XSP/JCL • Multi-node clustering ensures high availability and maximizes system usability • TP-Monitor – based system management • Provides JCL Analyzer • Integrated GUI based system admin via Open. Studio • Supports COBOL, PL/1, C, and Shell program • Dynamic Job Scheduling for load balancing • Spool information sharing • Provides spool backup system & catalog information change method. Integration Features Security Monitoring Features • Integration with an external scheduler • Integration with OS authentication. • Provides real-time I/O count on datasets. • Initiates batch jobs from online • Integration with Open. Frame/TACF to authenticate users & authorize the access to various resources of Open. Frame/Batch. • Monitoring of and statistics about JOB © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 10/40 • Supports various logs for analysis
3. Job Management 3. 1 Introduction 3. 2 JOB Flow 3. 3 JOB Scheduling 3. 4 JOB Group 3. 5 JOB Life Cycle 3. 6 Internal Reader 3. 7 Runner 3. 8 TEXTRUN 3. 9 TJES Servers 3. 10 SPOOL © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved.
3. 1 Introduction What is TJES (Tmax Job Entry Subsystem)? 1. Receives JOBs in a form of JCL 2. Schedules JOBs for processing 3. Controls JOB output processing TJES Components • TJES Engine - Consists of Tmax servers that control & monitor batch JOBs. - Manages & prints outputs. • Runner - Parses JCL statements and executes programs specified in the JCL. • TEXTRUN - Integrates with external, third party JOB scheduler (Control-M). - Submits a JOB and retrieves its result. • INTRDR - A module that can submit a JCL stream (used by OUTPUT processing server). - Used for executing a batch JOB from an online application. • TJESMGR - Command-line based user interface that allow users to manage JOBs. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 12/40
3. 1 Introduction 3. 1. 1 TJES Architecture Diagram © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 13/40
3. 2 JOB Flow Open. Frame Servers in Master Node User Interface ofrsasvr ofrlmsvr obmjschd Request logging 5. Request run 1. Job submit obmjhist 2. Authentication 14. Report done Open. Frame Servers in Each Node 4. Request scheduling obmjmsvr OF server 9. Dataset Lock / Un. Lock obmjinit 7. Execute runner 11. Signal terminated JCL Runner 10. Request printing Process JOBQ 3. Insert job info. into JOBQ © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 8. Update job as working 14/40 Request logging 6. Assign a job to a runner 13. Clear the runner slot shared memory (Runner Info) 12. Update job as finished
3. 2 JOB Flow 3. 2. 1 Macro Expand in TJES (XSP) ofrsasvr Open. Frame TJES User Interface 1. Macro Submit JCL parser obmjschd obmjhist 5. Expanded JCL parsing obmjmsvr 6. Authentication obmjinit 7. JOB scheduling … 2. Macro parsing JCL runner Macro parser OF server Process 3. Wait for a reply Library © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 15/40 ofrcmsvr 4. Request a reply & Reply a request tconmgr
3. 3 JOB Scheduling (MVS, MSP) TJES Scheduling Policy • Priority based Scheduling - The JOB with the highest priority is scheduled first (if a runner is available). - If multiple JOBs have the same priority, the JOBs are executed based on the order they were submitted. • Priority Aging - JOB priority may increase at regular intervals while the JOB is awaiting execution. • Node Affinity - Specify the Node where the JOB must be executed. • Matching Class of JOB and Runner - A runner can have multiple classes (up to 8) • Check JOBNAME duplicates - JOBs with duplicate JOBNAMEs cannot run simultaneously (in Working or Suspended) (optional). • Check Runner Status - A JOB can be assigned only if a runner is in ACTIVE status. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 16/40
3. 3 JOB Scheduling (XSP) TJES Scheduling Policy in XSP • Checks system multiplicity - In XSP, system multiplicity limits the number of jobs that can run simultaneously. • JOB Group level Scheduling - In XSP, a JOB group is selected. Next, a JOB in the JOB group is scheduled. - Priority based scheduling -Only when no JOBs in higher JOB Groups can be scheduled, JOB Groups with lower priority can be scheduled. • JOB level Scheduling - JOB level scheduling is affected by JOB group attributes (REL, Multiplicity). - Checks JOB Group Multiplicity - If REL == ‘Y’, JOBs are scheduled in first come first served order. If a JOB is in HOLD, the JOB group is also held. - If REL == ‘N’, JOB is scheduled by priority order. (submitted order in same priority) - Priority Aging - Node Affinity - Check JOBNAME duplicates - Checks runner status © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 17/40
3. 4 JOB Group (XSP) JOB Group • In XSP, a JOB must be contained in a JOB Group (default: SYSGRP). • Attributes of JOB Group - Multiplicity: the number of jobs executed concurrently. - REL: sequential scheduling of JOBs in the JOB group - Priority: JOB group priority. Used by scheduler - JOB default parameter: default parameter of the JOB in the JOB group. - WAIT: scheduling of the JOB group is postponed until -Time -Operator response -Data completion -The end of the other JOB Group © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 18/40
3. 5 JOB Life Cycle JOB Status © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. Description READY • Internal status during submission (intermediate status) START • JOB has been submitted but has not been allocated to a region HOLD • JOB has been submitted as TYPRUN=HOLD WORKING • JOB is currently being executed by JCL Runner • JOB can be paused using the SUSPEND command SUSPEND • State when a running JOB is suspended by a user • JOB can resume using the RESUME command DONE • JOB successfully completed • Result satisfies the condition ERROR • JOB successfully completed • Result does not satisfy the condition STOP • JOB is stopped by a user FLUSH • JOB cannot complete due to an error (JCL runtime parsing error, dataset related error or system error) 19/40
3. 6 Internal Reader • Internal Reader is special device that submits a JCL stream. • Ofrpmsvr (output processing server) uses Internal Reader to submit the output that is JCL. - Output of batch job - Output of online transaction • Example: //ddname DD SYSOUT=(oclass, INTRDR) © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 20/40
3. 7 Runner Role of Runner • Prepares dataset resources • Verifies resource authorization • Prevents any access conflicts using lock server • Sequentially executes program(s) specified in STEP • Checks for the JOB failure condition © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 21/40
3. 7 Runner 3. 7. 1 Runner Process (MVS) External Utility Internal Utility obmjinit Run each step in job : fork tjclrun Parsing Sort Fast. Unload FTP IDCAMS Fact Sql. Loader DSNTIAUL IEBGENER Application EXEC PROC JCL parser (Library) JCL EXEC PGM Parsing CALL Business Module (Shared Object) Procedure © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. PGMRTS 00 Business Module (EXE) 22/40
3. 7 Runner 3. 7. 1 Runner Process interacted with AIM and Macro Expand in Procedure (MSP) External Utility Internal Utility obmjinit Run each step in job: fork tjclrun Sort Fast. Unload FTP IDCAMS Fact Sql. Loader DSNTIAUL IEBGENER Application EXEC PROC Parsing JCL Macro parser (Library) EXEC PGM or PGM Using PED (Message File) OSAMFRUN JCL parser Parsing (Library) JXGIJSM © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. CALL Business Module (Shared Object) EXEC PGM Using PED (Presentation File) Parsing Expanded Procedure Application server Procedure 23/40 Application server CALL Business Module (Shared Library) (Shared Object)
3. 7 Runner 3. 7. 1 Runner Process interacted with AIM (XSP) External Utility Internal Utility obmjinit Run each step in job: fork tjclrun Sort Fast. Unload FCHK KQCAMS Fact Sql. Loader FCPY FDLT Application Parsing EXEC PGM or PGM Using PED (Message File) OSAMFRUN JCL Application server CALL Business Module (Shared Object) EXEC PGM Using PED (Presentation File) JXGIJSM © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 24/40 Application server CALL Business Module (Shared Library) (Shared Object)
3. 7 Runner 3. 7. 2 Runner Internal Changing User in Runner Case 1 1. STOP JOB 2. Kill child process Runner External Utility 2. Acquire Superuser Privilege Cannot kill child process when a Utility was executed by a different user Can kill child process because of root authority 3. Kill child process Runner 4. Return to normal User Running with authority of normal user Running with authority of super user (root) © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. Runner 25/40 External Utility
3. 7 Runner 3. 7. 2 Runner Internal Changing User in Runner Case 2 Runner 1. EXEC PGM (fork) Utility or Application 2. Change to authorized user 2. Use Resource Cannot access resource when the executed PGM does not have permission Data. Set Utility or Application with authorized user 3. Use Resource FILE Data. Set Utility or Application Running with non-authorized user Utility or Application Running with authorized user © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. Can access resource because PGM has permission 26/40 FILE
3. 8 TEXTRUN process • Third party JOB scheduler executes TEXTRUN with JCL script • TEXTRUN submits JCL Third party Scheduler • TJES executes Runner with JCL 1. Invoke with JCL • TEXTRUN is monitoring until JOB ends • TEXTRUN returns the JOB return code TJES • Third party JOB scheduler checks the return code and determines whether scheduling will continue or stop Restriction: TEXTRUN is monitoring only one JOB. JCL script must contain just one JOB. obmjmsvr 5. Return with return code 3. Execute JCL TEXTRUN 2. Submit JCL 4. Monitoring JOB and Get result © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 27/40 obmjinit TJCLRUN
3. 9 TJES Servers Functionality Type Instance Resources Managing JOB • Submits a JOB • Removes a JOB obmjmsvr • Provides information about JOBs • Manages the output of JOBs TCS Many / Node VSAM Scheduling JOB • Issues a unique JOBID to the newly submitted JOB • Schedules JOBs in the START status to runners obmjschd • Changes scheduling properties (class, status) of a JOB • Supports multiple runner classes • Boots and Shuts down TJES nodes (obmjinit) UCS One / Domain VSAM Executing JOB • Invokes Runner to execute the scheduled JOB. • Manages working JOBs (suspend, resume, stop) obmjinit • Manages runners (classes, status) • Monitors JOB status changes and updates JOBQ. • Boots and shuts down a TJES node (itself) Logging Name obmjhist Description • Logs TJES operations (JOB status changes, commands issued) © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 28/40 UCS One / Node TCS One / Domain VSAM/ Shared Memory N. A
3. 10 SPOOL What is SPOOL? 1. Special Dataset used by TJES 2. Stores internal data of TJES and outputs data 3. Backed up and restored by TJESMGR SPOOL Dataset • INPJCL - A replica of the submitted JCL • JESMSG - The execution information of JOB’s steps • JESJCL - A parsed tree of the submitted JCL • SYSMSG - A system message from TJCLRUN during JOB execution • CATPROC - A cataloged procedure in the submitted JCL • OUTPUT DATA - A output data defined by SYSOUT DD © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 29/40
4. TSO/CLIST 4. 1 TSO/E & CLIST © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved.
4. 1 TSO/E & CLIST Execution of CLIST at a terminal 4. Exec Perl clistrun 1. Exec CLIST Perl XSUB Open. Frame system Service Routine Interactive processing with user Receive the request and data from user at the terminal UNIX terminal Transmit dataset to and from PC 2. Parsing 3. converting CLST Scripts Perl Scripts IPF module Open. Frame module Data. Set PC with Open. Frame file transfer module © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. Spool Flow of control Flow of data 31/40
5. Features 5. 1 Security Feature 5. 2 Multi-Node Environment 5. 3 Failover © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved.
5. 1 Security Features 5. 1. 1 TACF - Structure and control flow TACF structure and control flow. Open. Frame ① ⑧ User request for resource access Resource Manager Data Set Tx. ⑦ Etc. ⑥ ② TACF Auth. Server ⑤ TACF API ④ ③ Authentication and access control information check Check access authority for the resource Grant or reject the resource access Resources resource TACF Repository © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. Send the request to TACF (RM) 33/40 Return the result of the service request
5. 1 Security Features 5. 1. 1 TACF – Key Functions ü User Authentication - A user ID allows a user to be recognized. A user password confirms the user ID is valid. Only users with a valid user ID and password are able to access the system. ü Resource Access Control - Resource Access Control protect system resources from being used by unauthorized users. ü Logging-in to Access Resources - The TACF records the daily log history of user logins and resource access based on the resource access level. - These log records are used as the source data for statistical reports and analysis of resource usage and user behavior. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 34/40
5. 1 Security Features 5. 1. 1 TACF – Key Features ü Provides various interfaces The TACF provides a Tmax based interface and an API type interface to meet customer security requirements as well as the needs of a unique customer environment. ü Flexible auditing level configuration The TACF can adjust the audit-level and generate log files for each resource access. Thus only specified log items remain and the overall log-overhead is reduced. ü Real-time resource management The TACF enables real-time resource management so system administrators can limit the use of resources by time and for every user. ü Memory caching The TACF’s memory caching feature minimizes the load on the DB. When cached information is modified, the caching memory is synchronized. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 35/40
5. 1 Security Features 5. 1. 2 Security in TJES • With OS account authentication - JCL userid and password are verified. • Integrated with TACF (Tmax Access Control Facility) - JCL userid and password are verified. - Surrogate submission is supported. - JOBNAME authorization check in JOB management is supported. -Submit, Start, Hold, Cancel, Stop, Suspend, Resume, and Remove a job. -View a Spool. © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 36/40
5. 2 Multi-node Environment Open. Frame Batch engine supports Multi-Node clustering to provide the necessary performance and stability for batch JOB processing. NODE A OFRLMSVR OFRSASVR OBMJSPBK OBMJHIST OFRSMLOG OBMJSCHD OFRPMSVR Shared Memory - Runner Info - Running JOB Info OFRCMSVR Shared Memory Runner Info - Running JOB Info OBMJMSVR Open. Studio/ Master-Agent UTILITY JCL RUNNER OBMJMSVR JCL RUNNER Tmax OFRUISVR TJESMGR JCL RUNNER OBMJINIT TJESEXTRUN OFRDMSVR OBMJINIT Control-M NODE B Tmax OFRUISVR OFRDSEDT DATASET MANAGEMENT LIBRARY TJESMGR UTILITY OFRDSEDT DATASET MANAGEMENT LIBRARY Open. Frame/TSAM Open. Studio Console TS 0 System Meta. Data Shared Disk 1 / Domain Server (TJES) DATASET SDS PDS GDGs SPOOL 1 / Domain Server (BASE) © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. TACF Authentication Information One / Node Server 37/40 TS 1 MCAT UCAT VVDS VOL 1 TS 2 JOBQ OUTPUTQ LOCK VVDS VOL 2 Many / Node Server TS 3 TSQ TS 4 VVDS VOL 3 VOL 4 ~ 12 Batch Process Library or Utility
5. 3 Failover TJES provides the following failover mechanisms based on Tmax TP-monitor: 1. TJES Level Failover – Reconnects to TSAM, tries to update the JOB status, and stops all executing jobs while obmjinit restarts 2. Server Level Failover – TMM (Tmax Management Server) automatically restarts processes that failed 3. Node Level Failover – A backup server automatically takes over when a failure occurs. The Node Level Failover is based on Tmax Multi-Node failover mechanism 4. DB Connection Failover – Switches to the backup connection when the primary DB connection fails © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 38/40
5. 3 Failover Server Level Failover Server Failure Server Restart CLL TMM obmjinit obmpmsvr ofrsasvr obmjinit CLH obmpmsvr obmjschd © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. obmjhist ofrsasvr 39/40 CLH obmjsvr obmjschd obmjhist
5. 3 Failover Node Level Failover Before Failure Node 1 Critical Server Batch Server Node 2 Data Batch Server After Failure Automatic Loading Node 1 Node 2 Data Batch Server Network Failure © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 40/40 Critical Server Batch Server Automatic reconnection
Thank you Tmax. Soft Co. , Ltd © 2013 Tmax. Soft Co. , Ltd. All Rights Reserved. 41/40
- Slides: 42