Workflow Group ACME Workbench Sterling Baldwin ACME Workflow

Workflow Group: ACME Workbench Sterling Baldwin ACME Workflow in the Browser Team Lead: Sterling Baldwin Sam Fries, Matt Harris, Bibi Raju, Lukasz Lacinski, Jim Mc. Enerney, Jeff Painter, Charles Doutriaux, Sahsa Ames

Project Goals • Entire ACME workflow in the browser

Project Goals • Entire ACME workflow in the browser – – – – Run the model Generate climatologies Run diagnostics Analyze and visualize output data Transfer files between facilities Archive to HPSS Publish to ESGF

Project Goals • Each part of the process works independently, and flows into the next step

Project Goals • Each part of the process works independently, and flows into the next step – Run the model -> Slurm/Moab and the ACME_script. csh

Project Goals • Each part of the process works independently, and flows into the next step – Run the model -> Slurm/Moab and the ACME_script. csh – Generate climatologies -> ncclimo

Project Goals • Each part of the process works independently, and flows into the next step – Run the model -> Slurm/Moab and the ACME_script. csh – Generate climatologies -> ncclimo – Run diagnostics -> metadiags

Project Goals • Each part of the process works independently, and flows into the next step – – Run the model -> Slurm/Moab and the ACME_script. csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT

Project Goals • Each part of the process works independently, and flows into the next step – – – Run the model -> Slurm/Moab and the ACME_script. csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus

Project Goals • Each part of the process works independently, and flows into the next step – – – Run the model -> Slurm/Moab and the ACME_script. csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus Archive to HPSS -> HTAR

Project Goals • Each part of the process works independently, and flows into the next step – – – – Run the model -> Slurm/Moab and the ACME_script. csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus Archive to HPSS -> HTAR Publish to ESGF -> esgf publication python module

Project Goals • Each part of the process works independently, and flows into the next step – – – – Run the model -> Slurm/Moab and the ACME_script. csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus Archive to HPSS -> HTAR Publish to ESGF -> esgf publication python module – That’s a lot of tools!

Workbench vision • Wrap each tool in a UI – – – Show the options for the tool (no manpage lookup required) Only allow correct input Automate where possible Each tool in one place Zero install on users machine

Workbench progress • Data Manager – Use Globus to transfer files from Edison – Import data from ESGF – Publish to our private ESGF staging node • Run Manager – Create, configure, and execute diagnostic jobs – Browse output – Upload to Diagnostic Viewer • Visualizations – Barebones VCS visualization of net. CDF files

Future work • ACME model run – Automatic post processing and data transfer, HPSS archive – Job chaining • job A triggers job B which triggers job C and D, or an error causes A to restart and email the error to the user. • Access controlled sharing for all user data – Share anything you want with just the people you want • Full visualization power of UV-CDAT – VCS in the browser

Demo