Big Data at the Natural Resource Ecology Laboratory

“Big Data” at the Natural Resource Ecology Laboratory Randall B. Boone Research Scientist, Natural Resource Ecology Laboratory and Associate Professor, Department of Ecosystem Science and Sustainability ISTe. C Big Data Forum Colorado State University April 18, 2013 Department of Ecosystem Science and Sustainability Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 1

Integrating across scales through Top-down and Bottom-up approaches D. Ojima Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 2

Agent-based Modeling and Big Data Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 3

G-Range, A Global Rangeland Model Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 4

T. Hilinski Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 5

Day. Cent and Century Simulations T. Hilinski Potential Vegetation: Mean Annual NPP (g. C/m 2) 1961 -2006 Mean Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 6

Managing Big Data at NREL Our data are big, but well described Unlike some industrial applications, the system must be exceedingly flexible Must be responsive to a variety of users (e. g. , more diverse uses than the ISTe. C Cray) Calculation-intensive uses Users are unlikely to have the ability to parallelize tools Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 7

Rubel Cluster – The Backbone of NREL’s “Mid-Performance Computing” Distributed memory computer cluster 256 processors 500 Gflops 463 GB memory 2 TB immediate storage 50 -60 TB extended storage, plus 120 TB coming online Private 1 Gbps Ethernet interconnect, with 10 Gbps access to storage Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 8

Rubel Cluster Provides a testing platform for the CRAY for some Others use alternatives, such as R, which can make use of multiple processors Good, but not high performance computing A mix of desktop and cluster analyses yield “midperformance computing” But it is effective! Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 9

Tom Hilinski Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 10

Rubel – Past, Present, and Future Apr 18, 2013 ISTe. C Big Data Forum, Colorado State University 11