Fast Data Analysis with Integrating Statistical Metadata in

Fast Data Analysis with Integrating Statistical Metadata in Scientific Datasets Jialin Liu, Yong Chen Data-Intensive Scalable Computing Laboratory (DISCL) Computer Science Department Texas Tech University Cluster’ 13

Scientific Applications Trend Ø Scientific simulations tend to be data intensive • VPIC, a plasma physic simulation code, generates 1 trillion particles, with 26 bytes per particle, and total size of 36 TBs for one file, each time step (source: LBNL) Data requirements for selected INCITE applications at ALCF PI Project Lamb, Don Fischer, Paul Dean, David Baker, David Worley, Patrick H. Wolverton, Christopher FLASH: Buoyancy-Driven Turbulent Nuclear Burning 75 TB Reactor Core Hydrodynamics 2 TB Computational Nuclear Structure 4 TB Computational Protein Structure 1 TB Performance Evaluation and Analysis 1 TB 5 TB Kinetics and Thermodynamics of Metal and Complex Hydride Nanoparticles Climate Science 10 TB Parkinson's Disease 2. 5 TB Plasma Microturbulence 2 TB Lattice QCD 1 TB Thermal Striping in Sodium Cooled Reactors 4 TB Gating Mechanisms of Membrane Proteins 10 TB Source: R. Ross et. al. , Argonne National Laboratory Washington, Warren Tsigelny, Igor Tang, William Sugar, Robert Siegel, Andrew Roux, Benoit On-Line Data. Off-Line Data Cluster’ 13 300 TB 5 TB 40 TB 2 TB 100 TB 345 TB 50 TB 10 TB 44 TB 8 TB 10 TB 2

Scientific Applications Trend (cont. ) Ø Collected data from instruments increases rapidly too • • • In a global climate model (left), with 100 × 120 km grid cell, PBs of data managed analyzed Various parameters, e. g. , temperature and wind speed, are recorded Scientists desire higher resolution and finer granularity, which can lead to even larger sizes of datasets Source: UCAR 3 Cluster’ 13

Traditional Databases Ø Traditional database has been successful • Structured and relational data for commercial applications • Transaction processing, non-procedural query analysis • Sophisticated indexing Ø Challenges for scientific applications • • Lack support for core scientific data types Lack support for rich access patterns No good visualization/plotting tools Performance maybe slow 4 Cluster’ 13

Scientific Datasets and Libraries Ø Scientific datasets and libraries, e. g. , HDF 5, Pnet. CDF, ADIOS, have been used for scientific data management. • • • Supports sophisticated data types and access patterns Programming interface for creating, accessing, and visualizing the data Integrated with parallel I/O Pnet. CDF Dataset Format Cluster’ 13 5

Recent Developments Ø Recent studies have started to utilize database techniques to integrate with scientific datasets to leverage both merits Ø Fast. Bit by Wu et. al. implements indexing on datasets Ø Fast. Query by Chou et. al. implements parallel indexing and query system for large-scale datasets Ø Su and Agrawal implemented user-defined subsetting Ø Scientific Data Service by Wu, Byna and Dong 6 Cluster’ 13

Idea and Motivation Ø Scientists are interested in understanding the phenomenon behind the data. A typical case is to select data points of interests by performing the range queries. Select data points From datasets Where pressure>80 And 12<temperature<25; Ø Traditionally, without any prior knowledge of the datasets, a costly process. Ø Our Idea: use statistical metadata to facilitate such data analysis. Ø The added metadata improve the query response by more than three folds Performance Comparison with and without statistics 7 Cluster’ 13

Fast Analysis with Statistical Metadata (FASM) and System Design System Architecture FASM system (Fast data Analysis with integrated Statistical Metadata) has four major components: Subsetting, Statistics Generating, Metadata Rich Datasets and Runtime. 8 Cluster’ 13

FASM Challenges Ø What type of subsetting scheme is better? Ø What type of statistical metadata is desired? Ø How to utilize the statistical metadata at the runtime? 9 Cluster’ 13

Dimension-Driven Subsetting Ø Subsetting refers to how to partition the datasets in order to integrate the statistical metadata. • For 3 D datasets, we can have 1 D, 2 D, 3 D and combined subsetting. Different Subsetting Schemes 10 Cluster’ 13

Locality-Driven Subsetting Ø The inconsistence between logical access and physical storage using scientific datasets causes locality issue. Locality in Subsetting Schemes Type Dimension Distance sub 1 (lat, lon) sub 2 (lon, level) (lat-1)×lon sub 3 (lat, level) lat×(lon-1) sub 4 (lon, time) (level×lat-1)×lon sub 5 (lat, time) (level×lon-1)×lat sub 6 (level, time) level×(lon×lat-1) 0 11 Cluster’ 13

Concurrency-Driven Subsetting Ø Concurrency plays a critical role in exploring parallelism in the access and analysis of scientific datasets. Scheme sub 1 sub 2 sub 3 sub 4, sub 5, sub 6 Concurrency min((x×m)/(stripe_size×leve l), n) min((x×m)/(stripe_size×lat), n) min((x×m)/(stripe_size×lon), n) min((x×m)/stripe_size, n) Distribution of Datasets on Parallel File Systems 12 Cluster’ 13

Statistics Generating and Enhanced Datasets Ø There are different statistics we can utilize. • e. g. , MIN, MAX, MEAN, MEDIAN, 5 -number statistics (min, lower quartile, median, upper quartile, and max) Ø A statistical metadata portion is added A Sample of Metadata Rich Datasets Cluster’ 13 13

FASM Runtime Ø The Runtime component leverages integrated statistical metadata to facilitate data analysis and queries. Input: query request and statistics_metadata; Read operation: Step 1: In each access, get Statistics_metadata; Step 2: Filter useless subsets; Step 3: Modify accessing pattern: new_start [] = FASM_start; new_count [] =FASM_count; Step 4: read: ncmpi_get_vara_float(ncid, varid, new_start[], new_count[], *fp); Return: Query Result An Example of Runtime Read Operation Ø The current FASM system is designed for write-once read-many type of applications, and thus does not deal with data modifications and regeneration of statistics. 14 Cluster’ 13

Current Evaluations Ø Testbed • Hrothgar, a 640 -node cluster at Texas Tech University • Each node contains two Intel Xeon (Westmere) 2. 8 GHz 6 -core processors with 24 GB of memory • Nodes are connected with DDR Infiniband • Pnet. CDF v 1. 3 Ø Datasets and Query • Randomly generated synthetic datasets, 300 KB-100 GB • Real application BCCR-BCM, 12 GB; • Randomly generated range query and analyses, e. g. , 10<pressure<30; 15 Cluster’ 13

Statistics and Performance Improvements Performance of Different Statistics • The proposed approach demonstrates clear performance advantages as the dataset size increases 16 Cluster’ 13

Locality and Concurrency Performance Regarding Locality of Various Subsetting Concurrency of Various Strip Size • Sub 1, sub 2 and sub 3 are better than sub 4 and • Sub 3 achieves the best performance when sub 5 schemes in terms of Locality. the strip size is 5 MBs, which is 1. 67 times • Confirms the equation prediction. faster than the worst case. 17 Cluster’ 13

Storage Overhead and Amortized Cost Storage Overhead of Added Metadata Amortized Cost • Storage overhead of integrated metadata is less than one percent for three subsetting schemes. 18 Cluster’ 13

Real Application Test BCCR Model Test Using FASM The BCCR-BCM 2. 0 model. This is a climate model, Bergen Climate Model (BCM), dataset downloaded from Bjerknes Center for Climate Research (BCCR). The size of the dataset ranges from 100 MB to 1. 8 GB, and the total size of these datasets is more than 12 GB. 19 Cluster’ 13

Related Work Ø Databases and Improvements • • Weakness of traditional database: not support derived types, e. g. , array; not support spatial access pattern; can not manipulate using standard application programs, etc. Fastbit provides a set of compressed bitmap indexes for scientific datasets. Su and Agrawal designed tools to support user-defined subsetting and aggregation over Net. CDF datasets. The proposed approach has a similar idea to the indexing scheme, but is lightweight and incurs significantly less storage overhead Query optimization technics Traditional Database Scientific Datasets 20 Cluster’ 13

Related Work (cont. ) Ø Existing Optimization for Scientific Datasets Management Libraries • • Pnet. CDF optimizes the serial Net. CDF using the technics of MPI-IO; Compression algorithms are utilized in HDF 5 to store the raw data. ADIOS supports collecting local, simple, statistical and/or analytical data Our FASM system tries to provide a systematic solution and provides integrating the statistical metadata during the initial write, which can be further utilized DATASETS 21 Cluster’ 13

Related Work (cont. ) Ø File Systems and Data Organizations for Scientific Datasets • • • Buck combined Map. Reduce and Net. CDF by implementing a Sci. Hadoop prototype Kalyanaraman mapped n-D datasets to the space-filling curves. This study introduces a new approach to boost data analysis performance, with considering file systems and the impact of data distribution and locality HILBERT CURVE MAP-REDUCE DATASETS HDFS PVFS LUSTRE 22 Cluster’ 13

Conclusion Ø The raw datasets and current formats not sufficient for achieving an optimal performance Ø Integration of database and HPC scientific datasets necessary • Potential has been shown in prior studies Ø We propose an idea of integrating statistical metadata into datasets Ø Experiments have shown the integrated statistical metadata improves the query and analysis performance 23 Cluster’ 13

Ongoing and Future Work Ø Integrate the FASM with Fast. Bit to form a two level query filtering. Ø Investigate further to support data modifications, resubsetting, and regeneration of statistics at runtime. 24 Cluster’ 13

Thank You Questions? Welcome to visit our website: http: //discl. cs. ttu. edu 25 Cluster’ 13