DATA and WORKFLOWS Data Life Cycle Discovery Upload

DATA and WORKFLOWS

Data Life Cycle Discovery Upload Data Commons Repository (DCR), Elasticsearch Discovery Environment, i. Commands, Cyberduck Metadata Publication Data Commons Repository (DCR), NCBI-SRA Add, delete, copy; metadata templates; bulk metadata Sharing Community Data folders, Data Commons, quick share links Analysis Discovery Environment, Atmosphere, Agave API, Bis. Que, DNA Subway

Access the Data Store Discovery Environment • Upload & download files <2 GB • Share, organize, edit data • Add metadata and templates • Publish data Cyber. Duck • Upload & download files of any size • Transfer data between Cy. Verse and other locations • Sync data with local drive POWER EASE i. RODs i. Commands • Upload & download files of any size • Share & organize data • Add metadata • Sync data with local drive

Discovery Environment Data Dissemination: NCBI SRA Submission Pipeline FASTQ file import and compression Create SRA Submission Packages from DE File menu Create / Update NCBI Bio. Projects 2 metadata templates Create NCBI Bio. Samples 10 metadata templates Enter SRA Library metadata 1 metadata template Validate package and transfer to SRA submission apps (Aspera Connect) User receives notification from SRA Can retrieve SRA report, correct, resubmit report retrieval app

Organize dataset Add Data. Cite metadata and Readme file Request DOI Validation by Data Commons curators Data available to public datacommons. cyverse. org Data Commons Discovery Environment Data Publication: Data Commons

Overview of Genomics Workflows Sequence Read Processing HTProcess Genome Assembly Transcriptome Assembly Analysis Genome Annotation Data Publication SRA Submission Data Commons Variation Analysis Association Pipeline Validate Pipeline Discovery Environment RNA-Seq Methylation Atmosphere Agave API

Aligners Top. Hat 2, STAR*, HISAT 2* Assembly Cufflinks 2, String. Tie* Differential expression Cuffdiff 2, DESeq 2*, Ballgown, edge. R, GFold Quantification RSEM, HTSeq, Bam_to_counts, kallisto, Salmon, Sailfish, e. Xpress Annotation Trinotate* Discovery Environment Atmosphere * New additions Both (DE & Atmo) Atmosphere images Discovery Environment RNA-Seq 1: Differential Expression

RNA-Seq 2: High Throughput Process Apps Discovery Environment For handling large groups of data and easier workflow management. Files are managed as a group or library contained in a single directory. HTProcess Read Cleanup Workflow Trimmomatic Fast. QC HTProcess Tuxedo Workflow HTProcess Top. Hat-2 HTProcess Cufflinks (HPC-Agave) Top. Hat 2 Cuff. Merge Discovery Environment Atmosphere * New additions Both (DE & Atmo) HTProcess Cuff. Diff

RNA-Seq 3: Multifactorial pairwise comparisons Discovery Environment For doing multiple pairwise comparisons of RNA-Seq data for differential gene expression analysis Read cleanup Trim-galore Target file Differential gene expression DESeq 2/edge. R (multifactorial pairwise comparisons) Aligner BWA Read counting HTSeq Raw counts Discovery Environment Atmosphere * New additions Both (DE & Atmo) DESeq 2 output edge. R output

Aligners Top. Hat-2. 1. 1 Evolinc-I output Linc. RNA Evolinc-II-2. 0 Assemblers Cufflinks-2. 2. 1 Merge Cuffmerge 2 Linc. RNA Evolinc-I-2. 0 Discovery Environment Atmosphere * New additions Evolinc-I output Both (DE & Atmo) Atmosphere image Discovery Environment Evolinc: Identification and Evolutionary analysis of Long Non-coding RNA (Linc. RNA)

Discovery Environment Genome Assembly HTProcess Read Cleanup Workflow Jellyfish Fast. QC Trimmomatic Fast. QC Whole Genome Assembly All. Paths. LG SOAPdenovo-2 Velvet Ray Newbler Spades Genome Assembly Analysis Assess assembly Get correctness via GAGE Discovery Environment Compute contig stats Atmosphere * New additions Snap-gene prediction Both (DE & Atmo) QUAST

Worker MAKER Worker Augustus SNAP Exonerate BLAST Repeat. Masker Worker Master instance Worker cctools i. Commands Worker In collaboration with the Douglas Thain lab (http: //www 3. nd. edu/~dthain/) MAKER output WQ-MAKER Jetstream Image Genome Annotation: WQ-MAKER in Jetstream

Expression Data Analysis Pipeline Discovery Environment by Cy. Verse. UK Differential Expression GP 2 S, Gradient Tool Clustering / Biclustering BHC, TCAP, Wigwams Network Inference CSI, h. CSI, o. CSI Transcription Factor Motif Enrichment HMT, MEME-La. B Discovery Environment Atmosphere * New additions Both (DE & Atmo)

Check mapping file + metadata validate_mapping_file Split libraries + quality filtering (Illumina) split_libraries_fastq Pick Operational Taxonomic Units (OTUs) pick_open_reference_otus Analyze diversity core_diversity_analyses Make Emperor - 3 D PCo. A plots make_emperor Discovery Environment Atmosphere * New additions Both (DE & Atmo) Atmosphere images Discovery Environment Metagenomics Using QIIME 1

Discovery Environment Transcriptome Assembly HTProcess. Read. Cleanup. Workflow Pipeline Jellyfish Trimmomatic Fast. QC Transcriptome Assembly Trinity SOAPdenovo - Trans Oases Bayesembler Cufflinks Transcriptome Assembly Analysis RNAQUAST GAGE to get correctness Discovery Environment Compute contig stats Atmosphere * New additions CD-HITEST Both (DE & Atmo) Annotate transcripts Decode transcript

Raw FASTA assessment, adapter removal, trimming Fast. QC + Scythe + Sickle De novo assembly of reads Trinity Assessment of assembly rna. QUAST Differential gene expression DESeq 2 or Edge. R Annotation of transcripts Trinotate Discovery Environment Atmosphere * New additions Both (DE & Atmo) Atmosphere images Discovery Environment Non-model organism de novo transcriptomics

Discovery Environment DNA Methylation Analysis Genome Indexer Bismark Genome Preparation Bisulfite Sequence Aligner and Methylation Caller Bismark BSMAP* GSNAP* Methylation Reporter Bismark Methylation Extractor ZED-align* Differential Methylation Caller Bisu. Kit* Discovery Environment Atmosphere * New additions Both (DE & Atmo)

Variant calling SAMtools, GBS, EMS Imputing NPUTE Population structure Structure, fast. Structure, PCA Association GLM, MLM, EMMAX, MLMM, PLINK, Fast. LMM, GEMMA*, Bayes. R* Discovery Environment Agave API * New additions Both (DE & Agave) Agave API Discovery Environment Association Analysis

Aligners BWA-MEM, BWA-ALN Reference prep Picard, SAMtools Variant callers GATK Unified. Genotyper, Platypus. py call. Variants, SAMtools mpileup Discovery Environment Agave API * New additions Both (DE & Agave) Agave API Discovery Environment Variant Caller Workflow

Validate Workflow Simulate Apps Industrial scale GWAS simulation files from Syngenta, Alpha. Sim Associate and Predict Apps Fa. ST-LMM, GEMMA, Plink, Qx. Pak, Bayes. R, Gen. Sel, Ridge. Regression Winnow App (calculate signal/noise) Python code, stats libraries, D. Hand’s metrics Demonstrate Apps (human-readable graphics and tables) Python and R code, R ggplot 2 Discovery Environment Agave API * New additions Both (DE & Agave) Agave API Discovery Environment Extensible, scalable testing of tool accuracy and precision

Raw reads assessment, trimming Genome in NCBI or FTP site Fast. QC, Scythe, Sickle Genome assembly Genome integration into Co. Ge All. Paths, Velvet, etc. Load. Genome annotation Integration of functional and diversity data with genome Maker Data publication Data Commons Discovery Environment Flow. Ge Syntenic analysis, visualize, compare to other sequences Syn. Map, Genome. View, GEvo Atmosphere Co. Ge Discovery Environment Comparative Genomics

Georeferencing and Alignment Geospatial Data Abstraction Library (GDAL) * Surface Models Geographic Resource Analysis Support System (GRASS) *, System for Automated Geospatial Analysis (SAGA) * Zonal Statistics Quantum Geographic Information System (QGIS) * Species Distribution Models (SDM) RStudio-Server (R) * Discovery Environment Atmosphere * New additions Both (DE & Atmo) Atmosphere images Discovery Environment Species Distribution Modeling

In the Works Phenomics workflow support • Image data management and analysis • Geographic information systems To discuss your phenomics workflow needs, contact info@cyverse. org.

Workflows: Quick Reference Guide Workflow Platform Limits Documentation All -- Data Commons Repository SRA Submission DE -- Documentation and tutorial RNA Seq 1 Differential Expression DE, Atmo -- Tutorial RNA Seq 2 HT Process DE, Agave 150 GB input data max Documentation RNA Seq 3 Multifactorial Pairwise Comparisons DE -- DESeq 2 tutorial edge. R tutorial Expression Data Analysis Pipeline (UK) DE -- Documentation Tutorial Evolinc DE, Atmo -- DE tutorial Atmo tutorial Genome Assembly DE, Agave -- Cleanup tutorial Assembly tutorial Genome Annotation WQ-MAKER Jetstream -- Tutorial Metagenomics QIIME DE, Atmo -- Atmo tutorial DATA Data Management Life Cycle GENOMICS DE - Discovery Environment Atmo - Atmosphere Agave - Agave API

Workflows: Quick Reference Guide (cont. ) Workflow Platform Limits Documentation Transcriptome Assembly DE, Agave 48 hrs run time max Tutorial Non-model Organism Transcriptomics DE, Atmo -- Methylation Analysis DE, Agave -- Bismark Genome Preparation Bismark Methylation Extractor Association Analysis DE, Agave -- Documentation and tutorials Variant Caller DE, Agave 48 hrs run time max Documentation Validate Atmo, Agave -- Documentation DE, Atmo -- GENOMICS (cont. ) GEOSPATIAL DATA Species Distribution Modeling DE - Discovery Environment Atmo - Atmosphere Agave - Agave API