Apache Spot Incubating A community approach to fighting
Apache Spot (Incubating) A community approach to fighting cyber threats © Cloudera, Inc. All rights reserved. 1
The hacker community collaborates everyday, it’s time we do the same. Services Products Training $500 $100 Free Hack a Corporate Email Account Angler Exploit Kits Learn to Crack Wifi © Cloudera, Inc. All rights reserved. 2
Status quo can’t keep up with the hacker community Silo Challenge Analytics Challenge Billions Endpoint Millions Events Trillions Scale Challenge Time Storing, processing, and analyzing 100 s of billions of events is not economically or technically feasible today Network User Integrating cross applications data for context and new analytics is not trivial Discovering unknown threats with advanced analytics (machine learning) is impossible on traditional systems © Cloudera, Inc. All rights reserved. 3
Apache Spot (Incubating) A community approach to fighting cyber threats. © Cloudera, Inc. All rights reserved. 4
Spot delivers… Scalable Platform with an Open Data Model 2. Analytic Collaboration Across the Community 3. Growing Application Ecosystem 1. … to address cybersecurity use cases. • Network Traffic Analytics • Threat Hunting • Incident Detection and Resolution • Cybersecurity Data Management • Custom Use Case © Cloudera, Inc. All rights reserved. 5
Custom Analytics Apache Spot (Incubating) Network Core Platform Endpoint User Packaged Analytics Cloudera • Apache Hadoop provides unrivaled data storage scale • Apache Spark provide large scale anomaly detection and advanced analytics • Cloudera provides data governance, security, and platform management Intel • CDH optimized for Intel hardware • Leverages Intel MPI library for application performance optimization • Data center compute power © Cloudera, Inc. All rights reserved. 6
Custom Analytics Apache Spot (Incubating) Network Open Data Models Endpoint User • Growing catalog of packaged ingestion pipelines for common data sources • Enriched events provide full context leading to better, faster analysis and decision making • Organizations maintain and control a single copy of their security data Packaged Analytics © Cloudera, Inc. All rights reserved. 7
Custom Analytics Apache Spot (Incubating) Network Packaged Analytics Endpoint User • Spot OSS includes machine learning algorithm for network traffic analytics • Emerging eco-system of ODM compliant vendor solutions • Additional OSS analytics will come from spot community Packaged Analytics © Cloudera, Inc. All rights reserved. 8
Custom Analytics Apache Spot (Incubating) Network Custom Analytics Endpoint User • Build custom analytics leveraging tools like Jupyter and Sense. io • Common data model across peers facilitates analytics collaboration • Leverage open source machine learning libraries (e. g. Mllib) Packaged Analytics © Cloudera, Inc. All rights reserved. 9
Cloudera Cyber based on Apache Spot and TAP Management (Director, Manager, Sentry, Navigator) Spot ODM Application Marketplace Spot ODM Analytics Analytic Services (Apache Spark, Sense. io, Jupyter) Spot Sample Data Sets Apache Spot Open Data Models (ODM) Ingestion (Kafka, Flume, Streamsets 1) Data Platform (CDH) Infrastructure (On Prem, AWS, Azure) ODM Compliant eco-system, both open source and ISV Network Traffic Analytics, Add’l OSS analytics Data Science workbench Community sourced, anonymized data sets for model development Logical and physical models Batch and Stream data ingestion Provisioning Management and Security Scalable storage and distributed processing Public or private clouds © Cloudera, Inc. All rights reserved. 10
Apache Spot (Incubating) Join the community that is fighting cyber threats. spot. incubator. apache. org © Cloudera, Inc. All rights reserved. 11
An overview of Apache Spot Flow Supervised Learning © Cloudera, Inc. All rights reserved. 12
An overview of Apache Spot DNS Supervised Learning © Cloudera, Inc. All rights reserved. 13
An overview of Apache Spot Proxy Supervised Learning © Cloudera, Inc. All rights reserved. 14
An overview of Apache Spot One out of a million © Cloudera, Inc. All rights reserved. 15
An overview of Apache Spot Open Data Models + SOLR © Cloudera, Inc. All rights reserved. 16
An overview of Apache Spot Open Data Models + SOLR © Cloudera, Inc. All rights reserved. 17
An overview of Apache Spot Investigate © Cloudera, Inc. All rights reserved. 18
An overview of Apache Spot Non Suspicious other than choice in news sources Investigate © Cloudera, Inc. All rights reserved. 19
An overview of Apache Spot Standard View – No Open Data Model Investigate © Cloudera, Inc. All rights reserved. 20
An overview of Apache Spot Investigate User Info (groups, creation dates, etc) + Suspicious info all in one place. Open Data Model Enrichment © Cloudera, Inc. All rights reserved. 21
An overview of Apache Spot Investigate Determine Incident Scope efficiently. Open Data Model Enrichment © Cloudera, Inc. All rights reserved. 22
Apache Spot (Incubating) Join the community that is fighting cyber threats. spot. incubator. apache. org © Cloudera, Inc. All rights reserved. 23
Thank you. © Cloudera, Inc. All rights reserved. 24
- Slides: 24