Plumbing the Computing Platforms of Big Data Dilma

Plumbing the Computing Platforms of Big Data Dilma Da Silva Professor & Department Head Computer Science & Engineering Texas A&M University

Where should I put host my big data app? • Systems == hardware + software enabled to run applications (+admins) • Opportunity for two-way collaboration: – You know which platform you want to use, but your application is not performing well in its target platform • Get resources where needed when needed – You are not sure about the right platform for the application – Your app can server as representative workload for evaluation system software research ideas 2/13/15 TAMU Big Data Workshop 2

Do I need large scale systems ? • Different communities, different requirements: – High End Computing / Super. Computing – Scale-out enterprise systems • But similar trends ? ! 2/13/15 TAMU Big Data Workshop 3

My app is running, now what? • ‘plug’ your app in the system, and it just works – Support for evolution • • • Efficiency: work done without wasting resources Elasticity: grow and shrink to adapt to demand Scale: can handle the ‘big’ part of your problem Agility Heterogeneity Resiliency: failures happens without loss of data or work • Problem determination: system helps you find your ‘bugs’, • Security, privacy • On the cheap 2/13/15 TAMU Big Data Workshop 4

Computing Industry Offerings (and where CSE people can help) • High End Computing (HPC, SC, commodity clusters) – Often application needs to be optimized for the platform • Large gap between domain and platform experts • Cloud computing platforms – So far optimized for enterprise systems 2/13/15 TAMU Big Data Workshop 5