Taverna SCAPE Taverna and my Experiment Tools for

Taverna SCAPE Taverna and my. Experiment Tools for creating and sharing workflows Alexandra Nenadic, David Withers University of Manchester Practical Tools for Digital Preservation: A Hack-a-thon York, 28 th September 2011

SCAPE What is a workflow? • Connecting a set of tools/services to create automated and repeatable processing/analysis

SCAPE Design and run workflows Taverna

SCAPE Taverna Features - Overview • Access to remote, distributed or local services and resources • Enables service interoperability and integration • Automates data flow between services • Implicit iteration over data sets, list handling and control links to determine order of service invocation • Extensible • Large external developer base • Various third party plugins available • Data and provenance collection

SCAPE Taverna Workbench • Graphical desktop tool • Drag-and-drop services into diagram • Connect services, run, reconnect, rerun • Integrates diverse set of tools

SCAPE Workflow Design Available services Tree view of the workflow structure Workflow diagram

SCAPE Taverna Workflows – Features in Detail • A set of (local and remote) services to analyze or manage data • Data-links connects services • • • i. e. output from service A is input to service B and C • Describes the desired dataflow instead of process coordination Nested workflows are also services Automatic iterations Parallelization Can customize list handling and control links • Fault tolerance • Retry (with delay and back off) • Failover (alternate services)

SCAPE Supported Services • • • SOAP/WSDL Web services REST Web services Soap. Lab Web services R statistical services Inline Beanshell scripts External tools and scripts (via ssh or localhost) Spreadsheet import XPath and text manipulation services SADI semantic Web services Nested workflows (workflow within workflow) Bio. Moby Bio. Mart • … your tool (write your own Taverna plugin)

SCAPE Workflow Results Progress report Previous runs Input data and results per port

SCAPE Workflow Provenance • Information about a workflow run • What happened? • And when? • Lineage tracing • Which input produced which output • Intermediate data • Inputs and outputs for each workflow step • Useful for debugging • Saved in standard format (such as OPM)

SCAPE Taverna is Domain-Independent • Bioinformatics • Biomedicine • Chemistry Pharmacogenomics Association study of Nevirapine-induced skin rash in Thai Population Systems Biology for Crop research, Bio. Diversity HIV and TB research in South Africa Sleeping Sickness in African Cattle

SCAPE Taverna is Domain-Independent • • Astronomy Data and text mining Digital content preservation (IMPACT) Social simulations Observing Systems Simulation Experiments JPL, NASA Astronomy & Helio. Physics Library Document Preservation British Library

SCAPE Share, discover and reuse workflows

SCAPE my. Experiment • http: //www. myexperiment. org • Social networking for people to share workflows and collaborate • Makes it easy for people to contribute to a pool of workflows, build communities and form relationships • Enables people to share, describe, reuse and repurpose workflows, reduce time-to-production, share expertise and avoid reinvention

SCAPE my. Experiment

SCAPE Workflow Sharing, Ownership and Attribution • my. Experiment can provide a central location for workflows from one community/group • my. Experiment allows you to say – Who can look at your workflow – Who can download your workflow – Who can modify your workflow – Who can run your workflow • Workflow ownership and attribution • Users do not need to start from scratch – reuse or modify existing workflows • Attribute/credit original author

SCAPE Use my. Experiment from Taverna

SCAPE Training • Tutorials and Training • 58+ tutorials to >900 people • >20 Universities, institutes and networks • Major conferences • Summer schools • Developer and User Days • Annotation Jamborees • Undergraduate and Postgraduate Bioinformatics in > 30 Universities

SCAPE Taverna and SCAPE • SCAPE preservation components/actions as services in Taverna workflows • Use Taverna Workbench to create and test SCAPE preservation workflows on local data • Then scale-up and run the workflows on a parallelized platform using Hadoop Map. Reduce • Share Taverna SCAPE workflows on my. Experiment