1 SHADOW DATABASES A PROBLEMS SOLVED B CONCEPT

1. SHADOW DATABASES A. PROBLEMS SOLVED. B. CONCEPT, THE COST. C. SETUP. D. SIMPLE SYNCHRONIZATION E. FLIP BETWEEN PRIMARY AND SHADOW F. DOING MAINTENANCE WITHOUT OUTAGES. G. INTERNALS and GOTCHA’s H. ONGOING SHADOW DATABASES I. OTHER CONCEPTS. 1

1. SHADOW DATABASES. A. PROBLEMS SOLVED AVOID OUTAGES 1. CONVERT FILES TO 4 B ISN OR REVERSE. 2. CHANGE BLOCKSIZE OF DATABASE 3. REORDER TO FORWARD INDEX COMPRESSION 4. COMPLETE REORDER (empty Index blocks, extents, cluster, resequ. ) 5. HIGH VOLUME TESTING. DBA TESTING 6. DATABASE INTEGRITY CHECKS. INTEGRITY FIX: ULD/LOD 7. CONVERT SPANNED FILES TO UNSPANNED 8. SOME MAJOR APPLICATION CHANGES 9. MERGE/SPLIT DATABASES 10. DISASTER RECOVERY SITE 2

B. CONCEPT, THE COST. PRIMARY DB 99 PLOG SHADOW DB 99 ADARES REGEN GAIN: MAXIMUM OUTAGE TIME: 5 – 15 MINS. COST: DUPE DASD FOR ASSO/DATA/WORK SETUP TIME MONITOR TIME ALTERNATIVE: EVENT REPLICATOR COST: SETUP TIME CPU LICENSE COST CPU: < 2% 3

C. THE SETUP. PRIMARY DB 99 SHADOW DB 99 DATASETS DB. P. ADA 099. ASSO DB. P. ADA 099. DATA DB. P. ADA 099. WORK DB. P. SEC 099. ASSO DB. P. SEC 099. DATA DB. P. SEC 099. WORK PLOGS, CLOGS, ETC. SVC NR = 238 SVC NR = 252 TASK NAME DB 099 NUC DB 099 SEC DDCARD ADARUN 99 SECRUN 99 4

C. THE SETUP. Cont. ADARUNS PRIMARY DB 99 SHADOW DB 99 CRITICAL DATABASE = 099 SVC = 238 LOCAL = NO PLOGRQ = YES DATABASE = 099 SVC = 252 LOCAL = YES MODE=MULTI DEFAULT / SMALL LFP=, LWP/LS=, NH=, LI=, LQ=, NQCID=, . . . OTHER NT = BIG (20) NAB = 16*NT (320) UTIONLY = YES/NO NU = BIG (20 K) LP = BIG (90) LWKP 2 = MIN (50) LBP / LFIOP = BIG ? OPENRQ = NO ? ? CT, TNAE ? CACHE, CBUFNO 5

D. SIMPLE SYNCHRONIZATION PRIMARY DB 99 BACKUP SHADOW DB 99 1. ADASAV SAVE: a. NOTE SYN 2 info and PLOG info 2. ADASAV RESTORE: a. ! USE ASSO/DATA/WORK names of SHADOW b. ! USE SVC of SHADOW 6

D. SIMPLE SYNCHRONIZATION P 1 PRIMARY DB 99 P 2 TASK: $PL COPY PLOG CATLG COPIED PLOGS JOBS: $RG REGEN PLOG SHADOW DB 99 DO STUFF 7

D. SIMPLE SYNCHRONIZATION ADARES DDKARTE: ADARES REGENERATE ADARES MTR=YES * FOR PERFORMANCE ADARUN PARALLELREAD * FOR FLEXIBILITY ADARES PLOGDBID=034 * ADARES PLOGNUM=2082 * FROM PLOG COPY DDDRUCK ADARES FILE=001, 002, 003. . . * FOR FLEXIBILITY ADARES FILE=016, 017, 018. . . : : : * ADARES CONTINUE * FOR DATABASE SYNCHRONIZATION ADARES DDKARTE (FIRST RUN): FROMCP=SYN 2 FROMBLK= * FROM ADASAV SAVE DDDRUCK 8

E. FLIP BETWEEN PRIMARY AND SHADOW 1. 2. 3. 4. ADAEND / HALT PRIMARY NUCLEUS WAIT TILL FINAL PLOG-COPY & REGENS COMPLETE HALT SHADOW NUCLEUS RENAME DATASETS, eg IDCAMS ALTER 'DB. P. ADA 099. ASSO' NEWNAME('DB. P. TMP 099. ASSO') ALTER 'DB. P. ADA 099. DATA' NEWNAME('DB. P. TMP 099. DATA') ALTER 'DB. P. ADA 099. WORK' NEWNAME('DB. P. TMP 099. WORK') ALTER 'DB. P. SEC 099. ASSO' NEWNAME('DB. P. ADA 099. ASSO') ALTER 'DB. P. SEC 099. DATA' NEWNAME('DB. P. ADA 099. DATA') ALTER 'DB. P. SEC 099. WORK' NEWNAME('DB. P. ADA 099. WORK') ALTER 'DB. P. TMP 099. ASSO' NEWNAME('DB. P. SEC 099. ASSO') ALTER 'DB. P. TMP 099. DATA' NEWNAME('DB. P. SEC 099. DATA') ALTER 'DB. P. TMP 099. WORK' NEWNAME('DB. P. SEC 099. WORK') 5. 6. START PRIMARY NUC START SHADOW NUC 9

F. DOING MAINTENANCE WITHOUT OUTAGES. KEY QUESTION: IS MAINTENANCE BEING DONE AT ET BOUNDARY OR NOT? AGAIN, IS MAINTENANCE BEING DONE AFTER ADASAV RESTORE? AGAIN, IS MAINTENANCE BEING BETWEEN PLOG REGENS? AGAIN, IS MAINTENANCE BEING DONE AFTER PLOG REGEN OF A PLOG FROM DATABASE BEING TERMINATED? 10

F. DOING MAINTENANCE WITHOUT OUTAGES. PLOG: BEFORE DS REC AFTER DS REC BEFORE DE/VALUE AFTER DE/VALUE ET PLOG END: AT THE END OF A REGEN, THE SHADOW DB MAY NOT BE AT: - ET BOUNDARY - COMMAND-LEVEL BOUNDARY EG. ADAVAL AFTER REGEN IS LIKELY TO SHOW ERRORS ! THE DATABASE IS ONLY SYNCHRONIZED AT 4 TIMES: 1. SYN 2 AT THE END OF AN ONLINE ADASAV 2. SYNC FROM THE HALT/ADAEND OF DATABASE 3. SYNC FROM THE OPERATOR COMMAND: SYNCC 4. SYNC-73 DBS TRANSACTIONS SUSPEND/RESUME 11

F. DOING MAINTENANCE WITHOUT OUTAGES. TO SYNC-UP THE SHADOW: SYN 2. RESTORE PRIMARY TO SHADOW AND DO NOT REGEN SYNC FROM HALT/ADAEND. REGEN LAST PLOG FROM SESSION. SYNC FROM OPERATOR COMMAND: SYNCC. 1. ISSUE SYNCC TO PRIMARY DATABASE (MAY CAUSE DELAY). 2. FEOFPL PRIMARY DATABASE. 3. BUILD REGEN JCL TO REGEN TOCP=SYNC FOR SYNCC. 12

F. DOING MAINTENANCE WITHOUT OUTAGES. RULES 1. DO NOT CHANGE ISNs ! 2. DO NOT MAKE DS CHANGES BASED ON DE ! (IF NOT SYNC’D) 3. RESPECT DE-VALUES AND FDT OPTIONS IN PLOG ! CAUTION: MAKE SURE ALL SHADOW MAINTENANCE USE THE SHADOW ADARUN SVC= ! MAKE SURE MAINTENANCE RUNS ON SAME LPAR ! 13

F. DOING MAINTENANCE WITHOUT OUTAGES. GOOD MAINT. EXAMPLES 1. IMPLEMENTING FORWARD INDEX COMPRESSION. 2. MOVING FILE X TO DASD PACK Y FOR PERFORMANCE. 3. USERS RECEIVING RESPONSE CODE 177, RUN ADAVAL. 4. FIX INTEGRITY ERROR VIA ADAULD/ADALOD. 5. CONVERTING FILE TO 4 -BYTE ISNS. 6. CONVERTING FROM EXPANDED FILES TO 4 -BYTE ISNS. 7. IMPLEMENTING LARGER BLOCKSIZE FOR PERFORMANCE. 8. TESTING RUN TIME OF ADACMP DECOMPRESS. 9. TESTING RUN TIME OF MAJOR APPLICATION CONVERSION. 14

F. DOING MAINTENANCE WITHOUT OUTAGES. BAD MAINT. EXAMPLES ! 1. ULD/LOAD FILE (USERISN=NO) ! ! 2. REORDER FILE BY DE ! ! 3. ADAINV ! (SEE ONLINE INV) ! 4. CHANGE COMPRESSION OPTIONS ! ! 5. CONVERT MUPEX TO 1 OR 2 (DATA, PE DE) 15

G. INTERNALS and GOTCHA’s SYNS UTILITIES ON PRIMARY. EXAMPLE. ADADBS NEWFIELD ADADBS ALLOCATE ADADBS RELEASE PLOG, REGEN SYNS CHECKPTS APPLIED PLOG, REGEN CC=4 ON REGEN JOB FILE EXCLUDED FROM REGEN PROCESS SYNP UTILITIES ON PRIMARY. EXAMPLE. ADAINV ADALOD 16

G. INTERNALS and GOTCHA’s IF YOU MUST SYNP ON YOUR PRIMARY OPTION A: 1. STOP THE 7 X 24 PROCESS. 2. DO SYNP FUNCTION ON PRIMARY. 3. BACKUP PRIMARY AND RESTORE TO SHADOW. 4. START REGEN FROM NEW SYN 2 CHECKPOINT OPTION B: 1. LOCK THE FILE FROM USE ON. 2. FEOFPL AND WAIT ON CORRESPONDING REGEN. 3. REMOVE THE FILE FROM THE REGEN FILE LIST. 4. PERFORM THE SYNP UTILITY ON BOTH PRIMARY AND SHADOW DB. 5. FEOFPL AND WAIT ON CORRESPONDING REGEN. 6. ADD THE FILE BACK TO THE REGEN FILE LIST. 8. UNLOCK THE FILE SO UPDATES CAN OCCUR. 17

G. INTERNALS and GOTCHA’s TO DO INTEGRITY CHECKS (eg. ADAVAL OPTION A: 1. RUN ADAVAL, NOTE ANY ERRORS. 2. FLIP PLOG ON PRIMARY AND DOING REGEN TO SHADOW. 3. RUN ADAVAL AND SEE IF ERRORS ARE THE SAME. OPTION B: 1. PUT THE DATABASE AT A SYNCHRONIZED POINT (SEE ABOVE) 18

USING THE SHADOW AS A RECOVERY DATABASE. WHY HOW JUST DO IT. 1 RISK: INTEGRITY ERRORS. IF STILL ONE PLOG TO BE REGENNED. 2 1. PUT /STOP/ IN 7 X 24 PROCESS. 2. MODIFY REGEN JOB TO RUN WITH ‘CONTINUE’ 3. REMOVE STOP, RUN LAST REGEN JOB. ALL PLOGS HAVE BEEN REGENNED. 3 1. COPY PLOG TO TAPE. 2. RUN ADARES BACKOUT OF PLOG ON TAPE. 3. RERUN LAST REGEN WITH ‘CONTINUE’. 19

H. ONGOING SHADOW P 1 PRIMARY DB 99 P 2 TASK: $PL COPY PLOG CATLG COPIED PLOGS JOB: $RG REGEN PLOG SECNDRY DB 99 RUNS 7 X 24 (OR NEARLY) 20

H. ONGOING SHADOW CONTROL DATASETS NEEDED FROM PLOG COPY (DDKARTE/CATLG) PLOGNAME PLOGNUM (DBID) : : : CONTROL STOPGO=STOP/GO LASTPLOG= CURRENTPLOG= (Special) CHECKPOINT NAME CHECKPOINT BLOCK 21

H. ONGOING SHADOW PLOG COPY PHILOSOPHY: QUICK AVOID DB LOCKUP PLOGSIZE (DUALPLS=): SHADOW FOR RECOVERY OR MAINTENANCE? WHEN ADASAV RUNS. FLIP PLOGs BY FEOFPLs OR SIZE? EXTRACT/WRITE DDDRUCK INFO TO DATASET EXTRACT/WRITE PLOGNAME TO DATASET SUBMIT REGEN JOB: IF NO “STOP” IF NO REGEN RUNNING IF PLOG NOT EMPTY 22

H. ONGOING SHADOW ADARES REGENERATE JOB PHILOSOPHY: KEEP SEPARATE FROM PLOG COPY PROCESS. RUN AS NEEDED. 1. DO FILE REGEN VS. DB REGEN: a. SPEED IS THE SAME. b. NO DATABASE-LEVEL LOCKING. c. FLEXIBILITY TO EXCLUDE FILES FROM REGENS. 2. MTR=YES ( 50% FASTER ) 23

H. ONGOING SHADOW ADARES REGENERATE JOB (3 STEP PROCESS) STEP CONTROL 1 1. IF /STOP/ END CC=2 2. READ DSN CONTROL 3. BUILD DDKARTE DSN (EG. CHKPT START) 4. UPDATE CONTROL LAST / CURRENT PLOG 5. END CC=0 STEP REGEN ADARES REGENERATE FILE LEVEL MTR=YES INPUT PLOG DSN INPUT DDKARTE DSN STEP CONTROL 2 1. UPDATE CONTROL: LAST PLOG 2. IF /STOP/ END CC=0 3. IF PLOGCPY RUNNING END CC=0 4. READ FOR NEXT PLOG: a. IF NONE, END CC=0 b. IF PLOG NULL, DELETE 5. SUBMIT REGEN JOB 24

H. ONGOING SHADOW DOING MAINTENANCE 1. PUT /STOP/ IN 7 X 24 PROCESS. 2. WAIT TILL ANY RUNNING REGEN COMPLETES. 3. DO MAINT ON SHADOW DB. 4. UNSTOP /STOP/ IN 7 X 24 PROCESS. 25

H. ONGOING SHADOW RESTORING FROM PRIMARY TO SHADOW 1. PUT /STOP/ IN 7 X 24 PROCESS. 2. WAIT TILL ANY RUNNING REGEN COMPLETES. 3. RESTORE SPECIFYING SHADOW DATASETS. 4. UPDATE CONTROL TO START AT SYN 2 CHECKPT. 5. UNSTOP /STOP/ IN 7 X 24 PROCESS. 6. SUBMIT REGEN JOB. 26

I. OTHER CONCEPTS 1. CONTINUOUS PLOG COPIES 2. ALL AS STARTED TASKS 3. REXX AS A TOOL 27