Towards Effective Virtualization of Intrusion Detection Systems Nuyun

Towards Effective Virtualization of Intrusion Detection Systems Nuyun Zhang†, Hongda Li†, Hongxin Hu†, and Younghee Park‡ † ‡ SDN-NFV Security 2017

Outline n Introduction n Motivations n Our Approach l Feasibility l IDS of our approach virtualization as microservice n Experiments l Benefits of scaling individually l Benefits of customization

Traditional IDSes n Single process IDS l Cannot scale to large volume of traffic n Parallelization of IDS l Clustered IDS: costly, fixed location/capacity l Multi-core/thread IDS: fixed capacity Traditional IDSes are inflexible

Virtualized IDS Based on NFV+SDN n New features l Dynamically create/destroy instances l Dynamically redistribute traffic n Current state l Split/Merge l Open. NF [NSDI’ 13] [SIGCOMM’ 15] Treat virtualized IDS as monolithic piece of software

Limitations of Monolithic Virtualized IDS (1/3) n Inefficient Resource Usage Available Resource Unusable Can’t fit Monolithic Instance Cloud

Limitations of Monolithic Virtualized IDS (2/3) n Hard Customization for different purposes Monolithic Virtualized IDS • Knowledge of source code • Error-prone • Implementation-dependent Customized IDSes

Limitations of Monolithic Virtualized IDS (3/3) n Inflexible Scalability Underloaded Detector 1 Overloaded Detector 2 Parser Detector 1 Detector 2 Scale-out Monolithic Virtualized IDS Underloaded Parser Monolithic Virtualized IDS Detector 1 Detector 2 Parser Monolithic Virtualized IDS

Our Approach IDS Virtualiaztion as Microservice n What is microservice? l Breaking application into independent smaller components l Components communicate through lightweight mechanisms n Why microservice? l Small and lightweight l Scale individually l Run independently

Feasibility of Our Approach Alerts, logs IDS High-level analysis Per-connection, simple events Traffic § Run independently § Communicate via welldefined messages Low-level per-connection parsing n Low-level per-connection parsing l Check-sum verification, stream reconstruction, pattern matching, etc. n High-level analysis l Cross-event detection tasks.

IDS Virtualization as Microservices n Decomposing of virtualized IDS l Low-level connection parsing service l High-level attack detecting service l Scheduler Parsing service 1 Traffic Scheduler Parsing service 2 Detecting service 1 Detecting service 2 Detecting service 3

Efficient Resource Usage Available Resource Unusable Microservice Instances Cloud

Easy Customization Microservices Customization for different purposes Monolith • • • Knowledge of source code Error-prone Implementation-dependent Customized IDSes • User-level service composing • Automation customizing • Implementation-independent

Flexible Scalability Parsing service 1 Detecting service 2 Traffic Scheduler Detecting service 3 Parsing service 2 Detecting service 4 Overloaded Individually scaling Agility and Efficiency

Experiments n Evaluate the benefits of: l Scaling individually l Customization n Simulation l No policy scripts Low-level parsing service l Selective policy scripts High-level detecting service n Real network traffic dataset l Mid-Atlantic Collegiate Cyber Defense Competition

Benefits of Scaling Individually Microservices CPU Usage for Microservices and Monoliths

Benefits of Scaling Individually Microservices Memory Usage for Microservices and Monoliths

Benefits of Customization Bro policy scripts customization Type of traffic Load All Customized scripts HTTPS Load all policy scripts Load only HTTPS scripts HTTP Load all policy scripts Load only HTTP scripts SSH Load all policy scripts Load only SSH scripts Others Load all policy scripts Load any other scripts

Benefits of Customization CPU Usage Type of traffic All scripts Customized Benefits HTTPS 30. 2% 28. 1% 7. 1% HTTP 29. 4% 26. 6% 9. 2% SSH 15. 8% 15. 0% 4. 7% Others 34. 2% 33. 2% 2. 8% Less CPU usage Memory Usage (MB) Type of traffic All scripts Customized Benefits HTTPS 1153. 5 1027. 5 10. 9% HTTP 721. 0 636. 0 11. 8% SSH 735. 0 654. 0 11. 0% Others 1252. 0 1087. 5 13. 1% Less Memory usage

Discussion and Future Work n Building Virtual IDS l Security challenge: multi-step attacks n Network Security as a Service l FW-as-a-Service l IDS-as-a-Service l New security NF-as-a-Service? n Io. T Security l Highly customizable l Agility

Q&A Thank you!

Back up n Limitation of our experiments n Microservices n Traffic volume variation

Limitations of Our Experiments n Bro uses disproportional CPU in lower traffic environment. n Benchmarking – Pseudo realtime. bro n In the future, send events to bro detector – capture_events n Metrics: includes memory and communication time n Real data from Internet/enterprise network

Microservices vs. Monoliths n Mircoservices: monitor all traffic Resource: R 1 Resource: R 2 Resource: R 3 • R 3: parsing service HTTP scripts All except HTTP scripts No scripts • R 1 -R 3: detecting service for HTTP Event Engine • R 2 -R 3: detecting service for non-HTTP All traffic n Monoliths: monitor all traffic Resource: R 5 HTTP scripts Event Engine HTTP traffic Resource Usage Resource: R 6 All except HTTP scripts Microservices Monoliths R 3 + (R 1 -R 3)+(R 2 -R 3) R 5+R 6 Event Engine Non-HTTP traffic

Microservices n Increasing concurrency and Dev. Ops requirements l Small services running in their own processes independently while communicating with each other through lightweight mechanisms l Breaking an application into smaller and completely independent components, enabling each component to scale individually and be available all the time

Traffic Volume Variation Gbps Expensive option: DDo. S attack on Feb. 2016 capacity ≥ peak traffic load 400 320 240 Significant Variation 160 80 0 2/19 2/22 2/25 Time Source: https: //blog. cloudflare. com/a-winter-of-400 gbps-weekend-ddos-attacks/