Robust REXX and Rules in OPS Systems Automation

Robust REXX and Rules in OPS Systems Automation Xchange - Sept. 24 -25, 1998 An independent consultant with Pro. Tech Inc. Telephone: USA (303) 652 -3900 Pro. Tech: USA (412) 373 -8855 Email: Mark. Pelt@peltind. com © Copyright 1998 Pelt Industries. All Rights Reserved.

Robust REXX and Rules in OPS Abstract: This session provides some suggestions on how to make your automation programs and rules strong, hardy, healthy and vigorous. The top 10 things you can do in your rules to make them more robust are discussed in detail.

Markus Pelt Consultant Pelt Industries Biography: Markus Pelt has been a mainframe developer for over 24 years. In 1985 he developed his first automation software package, now sold by Candle Corp. as AF/Operator. He was one of the initial developers of OPS/MVS in 1986, and wrote the original versions of MSF, AOF TEST, EPI and the OPS/REXX interpreter. He has also been involved in developing outboard console automation using Windows. Based in Boulder, Colorado, Mark is currently an independent consultant specializing in automation conversions, development and training. Education: Schools in Holland, Belgium, France and the USA B. S. Mathematics - Virginia Commonwealth University, Richmond, VA M. S. Operations Research - Stanford, California

Agenda n What’s Robust ? n Why Robust ? n Characteristics of Robust Programs n The top 10 things you can do in your rules to make them more robust

What’s Robust ? n STRONG Doesn’t fall over easily n HARDY Handles all conditions n HEALTHY Low maintenance diet n VIGOROUS Actively reports problems

Why Robust ? n Saves time – Maintenance time – Rerun time n Saves money – Cost of system unavailability – Cost of YOUR unavailability n Saves face – The code keeps working

Characteristics of Robust Programs Robust programs and rules – Rely little on hard-coded values – Use consistent and standardized routines – Avoid duplicate code – Make as few assumptions as possible – Are well documented for other programmers – Do not create more problems – Require little human intervention

The top 10 things you can do in your rules to make them more robust. 1. How to avoid hard-coding 6. What AOF TEST won't test 2. How to handle message format changes 7. How to avoid Auto-Enable killing your system 3. How to avoid timing dependencies 8. How to handle SQL errors 4. How to recover from unexpected conditions 5. Why adding comments doesn't waste time 9. How to handle EPI errors 10. How to handle crosssystem problems

How to avoid hard-coding To avoid hard-coding values: n n n Separate the data from your programs and rules Check any number or string in your program or rule as suspect If you must use them, put them in variables at top of your program or rule Change your code to: n n Use global variables for multiple rules/programs using same constants frequently Use tables for lists of constants n Use self-initializing code n Sanity check values

How to handle message format changes To lessen impact of minor message format changes: n n n Don’t use word or column positions n Avoid punctuation and capitalization assumption n Parse for key phrases Include sample message in documentation Don’t die quietly ! Report all detected message changes Error reporting should not create a new problem Consider impact of handling a message wrong

How to avoid timing dependencies Timing dependency = when event A must complete before event B starts n n n Do not issue “blind” commands Actions do not always complete in the same time Cross-system commands may have long delays CICS WTOR problem No response versus partial response Absence of evidence is not evidence of absence

How to avoid timing dependencies n n n Be willing to go the depth n Do live testing under differing circumstances n Challenge assumptions (we’ve always done it that way) Use SSM to define resource dependencies Check results of all actions where possible Don’t die quietly ! Report all errors Last resort - retry several times

How to recover from unexpected conditions You can’t check for specific unexpected conditions You can check for expected conditions n n Document assumptions made by your programs Check your assumptions manually when writing Have code check them too when it’s running Provide feedback on any false assumption(s)

Why adding comments doesn't waste time Without comments your code is like – interpreting smoke signals – reading tea leaves – tracking footprints – gazing in a crystal ball – looking at auras All possible but not easy

Why adding comments doesn't waste time A bug is a difference between what a program is intended to do versus what the program actually is doing. – The code defines what the program does. – Comments define what the program was intended to do. – Without comments, no one except the original programmer will know what’s a bug or the program’s purpose. – One minute writing a comment, can save hours reading and/or re-testing the code.

What AOF TEST won't test AOF TEST can test your rules against sample messages – from you – from OPSLOG AOF TEST won’t test: n message format errors if you manually enter your own n the right message if you automate on the wrong one n messages too rare to capture in OPSLOG n responses from dangerous “live” commands n assumptions about the arrival order of messages

How to avoid Auto-Enable from killing your system A rule that is auto-enabled will automatically activate on the next OPS startup n n If the rule causes a problem at IPL time, you may actually prevent the system from ever coming up again ! If you write a rule and enable it “live” without thorough testing, you are playing with fire If you write a rule and auto-enable it without thorough testing, you are playing with your job Use AOF TEST

How to handle SQL errors n SQL has many “gotchas” n Check for normal EOF first n Check RC & SQLcode return codes n Handle anticipated return codes n Use standard error check routine

How to handle SQL errors pgmname = “MYPROG” /* For error messages */ sqlcmd = “SELECT …” /* Setup SQL command */ address SQL sqlcmd /* Issue SQL command */ sqlrc = RC /* Save return code */ if SQLEOF(sqlrc, sqlcode, sqlcmd, pgmname) then do /* Handle EOF (no records found) */ end else if SQLERR(sqlrc, sqlcode, sqlcmd, pgmname, ”LOG”) then do /* Handle SQL error here */ end

How to handle EPI errors n Don’t assume session state n Check RC n Handle anticipated return codes n Use standard error check routine

How to handle EPI errors pgmname = “MYPROG” /* For error messages */ epicmd = “TYPE. . . ” /* Setup EPI command */ address EPI sqlcmd /* Issue EPI command */ epirc = RC /* Save return code */ if EPIERR(epirc, epicmd, pgmname, ”LOG”) then do /* Handle EPI error here */ end

How to handle cross-system problems Problems: Solutions: n MSF links down n Automatic retry of link n Long delays n Wait longer for response n No responses n Check return code n Reissue command (if possible) n Don’t assume other system alive n Use standard error check routine

Conclusion Coding robust rules and REXX n Saves time n Saves money n Saves face n Requires thinking ahead in terms of – reliability – maintainability – the programmer who has to read your code

QUESTIONS ÓCopyright 1994 Kokopelli Adventures

Notes