ADL Chain Run Script STAR AIT Weizhong Chen

ADL Chain Run Script STAR AIT Weizhong Chen December 11, 2014 11/1/2020 1

Overview The ADL Chain Run script is developed by the STAR AIT. Ø The ADL chain run script is mainly Perl script to automate the staging and processing of multiple JPSS SDR and EDR products. Ø The script calls ADL’s “run. ADLChain. Runner. pl” to process data. Ø The script uses a specified date with time range and/or a specified granule ID as an option to identify and stage input data. Ø 11/1/2020 2

Structure and function Script Description mruns_cm. sh Shell wrapper: Define the ADL location, date, start/end time, and/or granule ID chainrun_config. txt Configuration file of chain run script: define the input/output directory, case name, LUT location and Tiles location algorithm. list List of algorithms to be run. Upstream algorithms automatically processed in run_with. Metadata. pl Main chain run Perl script. Chain. Runs. pm Primary Perl module: Prepare run set for each granule, call ADL’s “run. ADLChain. Runner. pl” script to process data, then organize outputs. Link. Convert. H 5. pm Functions Perl module: include all functions to find and link RDR data from SCDR, then unpack them to ADL internal format as well as convert all ancillary data which include TLE, PW, NCEP and NAAPS. 11/1/2020 3

Build ADL Get an ADL snapview from Clear. Case in STAR’s network: 1. export $CSPP_HOME ($ADL_HOME= $CSPP_HOME/ADL) 2. source $ADL_HOME/build/env. Setup. sh 3. Build ADL: “$ADL_HOME/build. Adl. ksh” or “perl ${ADL_HOME}/script/build_adl. pl -clean -makefiles src -library -program -log $logfile” 11/1/2020 4

Run Chain Run Script Go to scripts location: “$CSPP_HOME/AIT_Scripts/ADL_Chain_Run/int_chainrun” 1. Modify “mruns_cm. sh”. 2. Modify “chainrun_config. txt”. 3. Modify “algorithm. list” file. (only keep the algorithms you want to run) Run “mruns_cm. sh algorithm. list &”. 11/1/2020 5

1. Modify “mruns_cm. sh” Ø # ADL location in Clear. Case: CM_ADL_Name="ADL 42_Mx 84_snap" export CSPP_HOME="/data 020/weizhong/CSPP_CM/ $CM_ADL_Name/home/pub/Clear. Case/STAR/JPSS/ADL" export ADL_HOME=$CSPP_HOME/"ADL" Ø # Processing Date and rough time range: Process. Date="2014 -05 -18" stime="t 0000000" etime="e 0805000" #gran. ID=“NPP 000810650886" Ø # Log file Location of Chain Run Script for each algorithms log_dir=“. /log" 11/1/2020 6

2. Modify chainrun_config. txt -p 1 Ø # -- user script location chainrun_dir=". " Ø # -- location to keep all inputs and output data. run_ADL_dir = "/data 095/weizhong/run_aerosol_dr 7723" Ø # -- unpacked ancillary data location run_ADL_dir_datasource "/data 020/weizhong/run_ADL" 11/1/2020 7

2. Modify chainrun_config. txt -p 2 Ø # -- Get the multiple inputs from the previous run’s “chainrun_config. txt” file in the following format. The default is empty. # input_dirs =“ 'algorithm name' 'User. Data' 'run_ADL_dir' ” # use ": " to separate different algorithms # use space to separate each items in every algorithm input_dirs = " " Ø # -- specific the folder's name of all results Case. Name = "aerosol_vcm" Ø # -- specific the folder to keep all links of h 5 files in the given time range. User. Data = "user_data" Ø # Granule Version Granule. VER = "A 1 " 11/1/2020 8

2. Modify chainrun_config. txt –p 3 Ø # All the ancillary source data location: including all Polar. Wonder, TLE, NCEP and NAAPS data. pub_datasource="/data 020/pub/NPP_DATA" Ø # Default LUTs location: $CSPP_HOME/LUTs # Run “$CSPP_HOME/AIT_Scripts/STAR_AIT_ADL_Setup. pl” to link all associated LUTs to your “$CSPP_HOME/LUTs”. # The user can replace them with their own LUTs under “$CSPP_HOME/LUTs”. All. Luts="LUTs: " Ø # location of all Tiles all_tiles_withmeta="/data 020/pub/NPP_DATA/TILES/Nov. Ground CCR 692 Tiles/with-meta-all" All those three directories have been prepared and maintained by STAR AIT. 11/1/2020 9

3. Modify “algorithm. list” ØCloud Mask: # alg. vcm VIIRS-EDR-Masks ØSurface Reflectance: # alg. surf VIIRS-EDR-Surf. Reflect ØVegetation Index: # alg. vi VIIRS-EDR-VI 11/1/2020 • • • • • • • • • • ATMS-REMAP ATMS-SDR Cr. IMSS-EDR Cr. IS-SDR OMPS-EDR-Np OMPS-EDR-Tc. Edr OMPS-EDR-Tc. Ip OMPS-SDR-Np. Cal OMPS-SDR-Np. Earth OMPS-SDR-Tc. Cal OMPS-SDR-Tc. Earth VIIRS-EDR-Aerosol VIIRS-EDR-Clouds 1 st VIIRS-EDR-Clouds 2 nd VIIRS-EDR-Cop VIIRS-EDR-ICh. Imagery VIIRS-EDR-Ice. Age VIIRS-EDR-Ice. Conc VIIRS-EDR-Ice. Qual VIIRS-EDR-Ist VIIRS-EDR-Lsa VIIRS-EDR-Lst VIIRS-EDR-MCh. Imagery VIIRS-EDR-Masks VIIRS-EDR-Ncc. Imagery VIIRS-EDR-Nhf VIIRS-EDR-OCC VIIRS-EDR-Snow VIIRS-EDR-Sst VIIRS-EDR-Surf. Reflect VIIRS-EDR-Surf. Type VIIRS-EDR-Surface. Albedo VIIRS-EDR-VI VIIRS-GIP-Gran. To. Grid. DSR VIIRS-GIP-Gran. To. Grid. Monthly. SRBTVI VIIRS-GIP-Gran. To. Grid. Snow. Ice. Cover VIIRS-SDR-Bright. Pixel VIIRS-SDR-Solar. Diffuser 10

Example one: VIIRS Cloud Mask Goal: • Generate common baseline Cloud Mask products for Aerosol team. • Show the next granule picks up the previous granule's results. • Show what happens if RDR data is missing.

Example one: Cloudmask 1 # Link and unpack RDRs, convert ancillary data, prepare run set for each granule. # 1 st granule: 0002 d 20140518_t 0603222_e 0604476_b 13235 Constructing Algorithm Chain. . . Construction Complete. Validating Algorithm Chain Prior to Run. . . Validation Complete. Algorithm Chain and Status: Pro. Sdr. Viirs. Controller: A 1: NPP 000810649179: Not Yet Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650032: Not Yet Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650886: Not Yet Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650032: Not Yet Built Delta Algorithm Chain Run Started. . . Running Algorithms With 2 Threads. 11/1/2020 12

Example one: Cloudmask 2 Algorithm Chain Queue: Pro. Sdr. Viirs. Controller: A 1: NPP 000810649179: Not Yet Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650032: Not Yet Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650886: Not Yet Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650032: Not Yet Built Creating Pro. Sdr. Viirs. Controller. exe TK file for granule ID NPP 000810649179 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Sdr. Viirs. Controller_NPP 000810649179. xml Running Pro. Sdr. Viirs. Controller. exe for granule ID NPP 000810649179: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Sdr. Viirs. Controller. exe Creating Pro. Sdr. Viirs. Controller. exe TK file for granule ID NPP 000810650032 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Sdr. Viirs. Controller_NPP 000810650032. xml Running Pro. Sdr. Viirs. Controller. exe for granule ID NPP 000810650032: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Sdr. Viirs. Controller. exe Success for Pro. Sdr. Viirs. Controller. exe with granule ID NPP 000810650032 Creating Pro. Sdr. Viirs. Controller. exe TK file for granule ID NPP 000810650886 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Sdr. Viirs. Controller_NPP 000810650886. xml Running Pro. Sdr. Viirs. Controller. exe for granule ID NPP 000810650886: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Sdr. Viirs. Controller. exe Success for Pro. Sdr. Viirs. Controller. exe with granule ID NPP 000810649179 Success for Pro. Sdr. Viirs. Controller. exe with granule ID NPP 000810650886 Creating Pro. Edr. Viirs. Masks. Controller. exe TK file for granule ID NPP 000810650032 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Edr. Viirs. Masks. Controller_NPP 000810650032. xml Running Pro. Edr. Viirs. Masks. Controller. exe for granule ID NPP 000810650032: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Edr. Viirs. Masks. Controller. exe Success for Pro. Edr. Viirs. Masks. Controller. exe with granule ID NPP 000810650032 Chain Run Completed Successfully! 11/1/2020 13

Example one: Cloudmask 3 # run_ADL_dir = "/data 095/weizhong/run_aerosol_dr 7723 “ # User. Data = "user_data" # Case. Name = "aerosol_vcm" linking. . . IICMO: AVAFO to user. Output. Dir: /data 095/weizhong/run_aerosol_dr 7723/data/output/user_d ata/with. Metadata/Pro. Edr. Viirs. Masks. Controller. Outputs/20140518 aerosol_vcm job_output_dir: /data 095/weizhong/run_aerosol_dr 7723/data/output/with. M etadata/Pro. Edr. Viirs. Masks. Controller. Outputs/20140518 aerosol_vcm/d 20140518_t 0603222_e 0604476_b 13235 NPP 000810650032 : VIIRS-EDR-Masks 11/1/2020 11. 717 min. 14

Example one: Cloudmask 4 # 2 nd granule: 0003 d 20140518_t 0604476_e 0606129_b 13235 Constructing Algorithm Chain. . . Construction Complete. Validating Algorithm Chain Prior to Run. . . Validation Complete. Algorithm Chain and Status: Pro. Sdr. Viirs. Controller: A 1: NPP 000810650032: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650886: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810651739: Not Yet Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650886: Not Yet Built Delta Algorithm Chain Run Started. . . Running Algorithms With 2 Threads. 11/1/2020 15

Example one: Cloudmask 5 Algorithm Chain Queue: Pro. Sdr. Viirs. Controller: A 1: NPP 000810651739: Not Yet Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650886: Not Yet Built Creating Pro. Sdr. Viirs. Controller. exe TK file for granule ID NPP 000810651739 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Sdr. Viirs. Controller_NPP 000 810651739. xml Running Pro. Sdr. Viirs. Controller. exe for granule ID NPP 000810651739: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Sdr. Viirs. Controller. exe Success for Pro. Sdr. Viirs. Controller. exe with granule ID NPP 000810651739 Creating Pro. Edr. Viirs. Masks. Controller. exe TK file for granule ID NPP 000810650886 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Edr. Viirs. Masks. Controller_N PP 000810650886. xml Running Pro. Edr. Viirs. Masks. Controller. exe for granule ID NPP 000810650886: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Edr. Viirs. Masks. Controller. ex e Success for Pro. Edr. Viirs. Masks. Controller. exe with granule ID NPP 000810650886 Chain Run Completed Successfully! 11/1/2020 16

Example one: Cloudmask 6 # link all final HDF 5 products to user. Output. Dir directory linking. . . IICMO: AVAFO to user. Output. Dir: /data 095/weizhong/run_aerosol_dr 7723/data/output/ user_data/with. Metadata/Pro. Edr. Viirs. Masks. Controller. Outputs /20140518 -aerosol_vcm job_output_dir: /data 095/weizhong/run_aerosol_dr 7723/data/output/ with. Metadata/Pro. Edr. Viirs. Masks. Controller. Outputs/20140518 -aerosol_vcm/ d 20140518_t 0604476_e 0606129_b 13235 NPP 000810650886 : VIIRS-EDR-Masks 11/1/2020 5 min. 17

Example one: Cloudmask 7 # Failed granule (missing RDR data): 0103 d 20140518_t 0227090_e 0228344_b 13233 …… Algorithm Chain and Status: Pro. Sdr. Viirs. Controller: A 1: NPP 000810519447: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810520300: Unable to Build Pro. Sdr. Viirs. Controller: A 1: NPP 000810521154: Not Yet Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810520300: Not Yet Built …. . . Creating Pro. Sdr. Viirs. Controller. exe TK file for granule ID NPP 000810520300 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base/log/Pro. Sdr. Viirs. Controller_NPP 000810520300. xml Running Pro. Sdr. Viirs. Controller. exe for granule ID NPP 000810520300: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base/bin/Pro. Sdr. Viirs. Controller. exe Creating Pro. Sdr. Viirs. Controller. exe TK file for granule ID NPP 000810521154 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base/log/Pro. Sdr. Viirs. Controller_NPP 000810521154. xml Running Pro. Sdr. Viirs. Controller. exe for granule ID NPP 000810521154: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base/bin/Pro. Sdr. Viirs. Controller. exe Algorithm Run Failure - Pro. Sdr. Viirs. Controller, NPP 000810520300 Failed During Execution! Algorithm Run Failure - Pro. Sdr. Viirs. Controller, NPP 000810520300 Failed to Produce an Output Product! Error: An Algorithm in the Chain has Failed to Complete Successfully! Waiting for Additional Algorithm Threads to Finish Before Exiting! Success for Pro. Sdr. Viirs. Controller. exe with granule ID NPP 000810521154 Chain Run did not Completed Successfully!

Example Two: Surface Reflectance Goal: • Generate a downstream for aerosol team. • Utilize previous VCM results to speed up the run.

Example two: Surf. Reflect 1 Modify chainrun_config. txt: Ø Modify “input_dirs” variable in chainrun_config. txt to include previous results. Get them from VCM run’s chainrun_config. txt and algorithm. list files. # input_dirs =“ 'algorithm name' 'User. Data' 'run_ADL_dir' : ” input_dirs="VIIRS-EDR-Masks aerosol_vcm /data 095/weizhong/run_aerosol_dr 7723” Ø Case. Name = “aerosol_base" 11/1/2020 20

Example two: Surf. Reflect 2 NPP 000810650886 Construction Complete. Validating Algorithm Chain Prior to Run. . . Validation Complete. Algorithm Chain and Status: Pro. Sdr. Viirs. Controller: A 1: NPP 000810649179: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650032: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650886: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810651739: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810652593: Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650032: Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650886: Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810651739: Built Pro. Edr. Viirs. Aerosol. Controller: A 1: NPP 000810650886: Not Yet Built Pro. Edr. Viirs. Surf. Reflect. Controller: A 1: NPP 000810650886: Not Yet Built Delta Algorithm Chain Run Started. . . Running Algorithms With 2 Threads. 11/1/2020 21

Example two: Surf. Reflect 3 Algorithm Chain Queue: Pro. Edr. Viirs. Aerosol. Controller: A 1: NPP 000810650886: Not Yet Built Pro. Edr. Viirs. Surf. Reflect. Controller: A 1: NPP 000810650886: Not Yet Built Creating Pro. Edr. Viirs. Aerosol. Controller. exe TK file for granule ID NPP 000810650886 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Edr. Viirs. Aerosol. Controller_NPP 00 0810650886. xml Running Pro. Edr. Viirs. Aerosol. Controller. exe for granule ID NPP 000810650886: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Edr. Viirs. Aerosol. Controller. exe Success for Pro. Edr. Viirs. Aerosol. Controller. exe with granule ID NPP 000810650886 Creating Pro. Edr. Viirs. Surf. Reflect. Controller. exe TK file for granule ID NPP 000810650886 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/log/Pro. Edr. Viirs. Surf. Reflect. Controller_NP P 000810650886. xml Running Pro. Edr. Viirs. Surf. Reflect. Controller. exe for granule ID NPP 000810650886: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base_a 1/bin/Pro. Edr. Viirs. Surf. Reflect. Controller. exe Success for Pro. Edr. Viirs. Surf. Reflect. Controller. exe with granule ID NPP 000810650886 Chain Run Completed Successfully! 11/1/2020 22

Example two: Surf. Reflect 4 linking. . . IVISR to user. Output. Dir: /data 095/weizhong/run_aerosol_dr 7723/data/outp ut/user_data/with. Metadata/Pro. Edr. Viirs. Surf. Reflect. Controll er. Outputs/20140518 -aerosol_base job_output_dir: /data 095/weizhong/run_aerosol_dr 7723/data/outp ut/with. Metadata/Pro. Edr. Viirs. Surf. Reflect. Controller. Outputs/ 20140518 aerosol_base/d 20140518_t 0604476_e 0606129_b 13235 NPP 000810650886 : VIIRS-EDR-Surf. Reflect 11/1/2020 2 min. 23

Third Example : Vegetation Index Goal: • Generate VI products for VI group. • Fix the RDR missing problem. • Add both previous VCM and Surface Reflectance results to current run.

Example three: VI 1 Modify chainrun_config. txt: Ø # Modify “input_dirs” variable in “chainrun_config. txt” to include previous results from VCM and Surf. Reflect: input_dirs = "VIIRS-EDR-Masks aerosol_vcm /data 095/weizhong/run_aerosol_dr 7723: VIIRS-EDR-Surf. Reflect aerosol_base /data 095/weizhong/run_aerosol_dr 7723" Ø # Modify inputs and outputs location. run_ADL_dir ="/data 048/pub/weizhong/temp/run_viirs_vi" Ø Case. Name = "vi_base" 11/1/2020 25

Example three: VI 2 Link RDR data: # Before link RDR data: scdr-files: No files found The RVIRS-RNSCA already expired on SCDR for 2014 -05 -18!!! Please link your own data to the following datasource directory: /data 048/pub/weizhong/temp/run_viirs_vi/HDF/RVIRSRNSCA/2014 -05 -18 # After link RDR data: (get RDR data from external source, like PEATE: ftp: //peate. ssec. wisc. edu/all. Data/ingest/ ) The data has been linked from SCDR previously! Please check datasource here: /data 048/pub/weizhong/temp/run_viirs_vi/HDF/RVIRSRNSCA/2014 -05 -18 11/1/2020 26

Example three: VI 3 0003 d 20140518_t 0604476_e 0606129_b 13235 NPP 000810650886 Constructing Algorithm Chain. . . Construction Complete. Validating Algorithm Chain Prior to Run. . . Validation Complete. Algorithm Chain and Status: Pro. Sdr. Viirs. Controller: A 1: NPP 000810649179: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650032: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810650886: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810651739: Built Pro. Sdr. Viirs. Controller: A 1: NPP 000810652593: Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650032: Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810650886: Built Pro. Edr. Viirs. Masks. Controller: A 1: NPP 000810651739: Built Pro. Edr. Viirs. Aerosol. Controller: A 1: NPP 000810650886: Built Pro. Edr. Viirs. Surf. Reflect. Controller: A 1: NPP 000810650886: Built Pro. Edr. Viirs. VI: A 1: NPP 000810650886: Not Yet Built Delta Algorithm Chain Run Started. . . Running Algorithms With 2 Threads. Algorithm Chain Queue: Pro. Edr. Viirs. VI: A 1: NPP 000810650886: Not Yet Built Creating Pro. Edr. Viirs. VI. exe TK file for granule ID NPP 000810650886 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base/log/Pro. Edr. Viirs. VI_NPP 000810650886. xml Running Pro. Edr. Viirs. VI. exe for granule ID NPP 000810650886: A 1 /data 020/weizhong/ADL 4. 1/CSPP/ADL 4. 2_mx 8. 4_base/bin/Pro. Edr. Viirs. VI. exe Success for Pro. Edr. Viirs. VI. exe with granule ID NPP 000810650886 Chain Run Completed Successfully! 11/1/2020 27

Example three: VI 4 # run_ADL_dir = "/data 048/pub/weizhong/temp/run_viirs_vi" # Case. Name = "vi_base" linking. . . VIVIO to user. Output. Dir: /data 048/pub/weizhong/temp/run_viirs_vi/data/outp ut/user_data/with. Metadata/Pro. Edr. Viirs. VIOutputs/20140518 -vi_base job_output_dir: /data 048/pub/weizhong/temp/run_viirs_vi/data/output/wit h. Metadata/Pro. Edr. Viirs. VIOutputs/20140518 vi_base/d 20140518_t 0604476_e 0606129_b 13235 NPP 000810650886 : VIIRS-EDR-VI 11/1/2020 0. 65 min. 28

Time Consumption for one granule VIIRS-SDR: 4. 40 min. VIIRS-SDR-Bright. Pixel: 15. 80 min. VIIRS-SDR-Solar. Diffuser: 5. 13 min. VIIRS-EDR-ICh. Imagery: 17. 48 min. VIIRS-EDR-MCh. Imagery: 14. 30 min. VIIRS-EDR-Ncc. Imagery: 14. 03 min. VIIRS-EDR-Masks: 14. 03 min. VIIRS-EDR-Cop: 17. 68 min. VIIRS-EDR-Clouds 1 st: 21. 75 min. VIIRS-EDR-Clouds 2 nd: 41. 23 min. VIIRS-EDR-Aerosol: 25. 70 min. VIIRS-EDR-Ice. Qual: 29. 95 min. VIIRS-EDR-Ice. Conc: 54. 41 min. VIIRS-EDR-Ist: 54. 93 min. VIIRS-EDR-Ice. Age: 55. 36 min. VIIRS-EDR-Snow: 33. 11 min. VIIRS-EDR-Surf. Reflect: 32. 91 min. VIIRS-EDR-VI: 28. 00 min. VIIRS-EDR-Surf. Type: 32. 76 min. VIIRS-EDR-Lst: 31. 85 min. VIIRS-EDR-Lsa: 55. 45 min. VIIRS-EDR-Sst: 56. 10 min. VIIRS-EDR-OCC: 55. 23 min. VIIRS-EDR-Nhf: 57. 98 min. VIIRS-EDR-Surface. Albedo: 59. 43 min. System info: - Machine rhw 3022: 16 core 3. 2 GHz cpu and 32 G mem. 11/1/2020 Notes: - Time is approximate and varies from run to run - Time indicates processing starting from RDR 29

Time consumption for one day Algorithm Time Consumption VIIRS Cloud Mask 5485 min ~ 4 days Surface Reflectance 2264 min ~ 38 h Vegetation Index 660 min ~ 11 h Time required to run a full day’s worth of data, assuming previous steps in chain are complete. 11/1/2020 30

Run old data by using new MX build Ø Link RDR data to script’s default input location: For example, for VIIRS: $run_ADL_dir/HDF/RVIRS-RNSCA/2014 -05 -18 Ø Change “Effectivity” date: 1. Replace all $CSPP_HOME/LUTs with your own LUTs. 2. Change starting “Effectivity” date in all metadata of related LUTs including upstream algorithms. 11/1/2020 31

Example: Problem: A user tries to run Omps Tc. Earth SDR algorithm on an older date 2013 -02 -05 using the current Mx 8. 6 build. § The first run failed. Error message: “/log/Pro. Sdr. Omps. Tc. Earth. Controller_20141110_190845_26999. log: 2014/11/1 0 19: 08: 52. 591. 070 (26999. 47951017882304): DBG_HIGH Pro. Cmn. Algorithm. cpp|7933|Error retrieving data for OMPS-TC-BIAS-GND-PI” Solution: After investigating its corresponding “*. asc” meta data of “OMPS-TC-BIAS-GND-PI” LUT, we noticed : ("Effectivity" DATETIMERANGE EQ "2014 -01 -23 13: 30: 00. 000000" "2029 -12 -31 23: 59: 57. 000000") § After change Effectivity date from "2014 -01 -23" to "201301 -23” in all related LUTs, it run successfully. 11/1/2020 32

Contributions We have successfully processed one day’s run for several science teams. In all cases, we provided them both baseline results and results with all their changes. Ø A Golden day test on date 11/10/2012 for Cloud Mask team. Ø One day global test for Aerosol team to investigate changes in DR 7723 and other DRs. Ø One day global test for VI team with J 1 Upper modifications (DR 7216). Ø One day global test for OMPS team. 11/1/2020 33

Summary We have: • Developed and utilized an ADL chain run script. • Demonstrated the script capability and usefulness by assisting several algorithm development teams. • Included both the script and documentation in Clear. Case for configuration management • Demonstrated ability to keep pace with new MX build releases. By using Clear. Case with the ADL chain run script, running ADL becomes an easy job. Within STAR’s network, a user can stage ADL chain runs for any algorithm and any date and time and begin getting results in minutes. 11/1/2020 34

Thank You! 11/1/2020 35