Towards Efficient Traffic Monitoring for Science DMZ with

Towards Efficient Traffic Monitoring for Science DMZ with Side-Channel based Traffic Winnowing Hongda Li, Fuqiang Zhang, Lu Yu, Jon Oakley, Hongxin Hu, and Richard R. Brooks SDN-NFV Security 2018

Outline n Introduction n Approach Overview n Lightweight Detection System Development n Evaluation

Traditional High Performance Data Transfer Campus Router/Firewall WAN Campus LAN Data Servers

Traditional High Performance Data Transfer 1. Inappropriate security policies Campus Router/Firewall WAN Campus LAN 2. Low-performance general-purpose networks 3. Hard to monitor performance Data Servers Campus LAN 4. Low-performance general-purpose data servers Data Servers

Science DMZ Campus Router/Firewall Border Router WAN Border Router Virtual/Dedicated Connection Science DMZ Router Campus LAN High performance Data Transfer Nodes

Threat Model Border Router WAN Traffic monitoring to capture the anomalies Science DMZ Router Compromised Data Transfer Node Malicious programs are installed and launched

Tension Between Two Requirements Border Router WAN Traffic monitoring to capture the anomalies n High-performance data transfer l 100 Gbps or higher n Low-performance traffic monitoring ● Snort with 800 Mbps per processor ● Bro with 80 Mbps per core Compromised Data Transfer Node Malicious programs are installed and launched

Limitations of Existing Approaches n Improving the processing capacity of IDSes l Multi-thread/core l Cluster-based l Special IDSes hardware IDSes

Limitations of Existing Approaches n Improving the processing capacity of IDSes l Multi-thread/core l Cluster-based l Special IDSes hardware IDSes Devoting more resources

Limitations of Existing Approaches n Improving the processing capacity of IDSes l Multi-thread/core l Cluster-based l Special IDSes Devoting more resources hardware IDSes n Tuning the IDSes based on traffic pattern l Predicting the traffic pattern

Limitations of Existing Approaches n Improving the processing capacity of IDSes l Multi-thread/core l Cluster-based l Special IDSes Devoting more resources hardware IDSes n Tuning the IDSes based on traffic pattern l Predicting the traffic pattern Lack of dynamics

Limitations of Existing Approaches n Improving the processing capacity of IDSes l Multi-thread/core l Cluster-based l Special IDSes Devoting more resources hardware IDSes n Tuning the IDSes based on traffic pattern l Predicting the traffic pattern n Sci. Pass l Totally bypassing IDSes Lack of dynamics

Limitations of Existing Approaches n Improving the processing capacity of IDSes l Multi-thread/core l Cluster-based l Special IDSes Devoting more resources hardware IDSes n Tuning the IDSes based on traffic pattern l Predicting the traffic pattern Lack of dynamics n Sci. Pass l Totally bypassing IDSes Lack of visibility after bypassing

Overview of Our Approach Continuous monitoring Border Router Lightweight Detection System WAN Update flow rules Open. Flow switch Reflected Traffic Filtered Traffic … IDS Instances Virtualization Platform High Performance Data Transfer Nodes Dynamically adapting to load variations

Lightweight Detection System Requirements n Very low false negative rate l Malicious l Benign traffic is rarely considered as benign traffic can be considered as malicious n Efficient detection l Process each packet faster l Analyze only a small portion of traffic to filter

Insight #1 of Science DMZ n Domain-specific applications l Lustre/GPFS (high-speed parallel file system) l Grid. FTP/FTD (high-speed data transfer) l XRoot. D (discipline-specific tools)

Insight #1 of Science DMZ n Domain-specific applications l Lustre/GPFS (high-speed parallel file system) l Grid. FTP/FTD (high-speed data transfer) l XRoot. D (discipline-specific tools) Baseline of what traffic should be deemed as benign

Insight #1 of Science DMZ n Domain-specific applications l Lustre/GPFS (high-speed parallel file system) l Grid. FTP/FTD (high-speed data transfer) l XRoot. D (discipline-specific tools) Baseline of what traffic should be deemed as benign Side-Channel Analysis Model of benign traffic

Modeling Network Protocols n Network protocol modeling via Hidden Markov Models (HMMs) l C. Lu, et al. “A normalized statistical metric space for hidden markov models. ” l L. Yu, et al. “Inferring statistically significant hidden markov models. ” l X. Zhong, et al. “Side channel analysis of multiple pmu data in electric power systems. ” It’s been proved feasible to model the network protocol by time intervals of packets

Insight #2 of Science DMZ n Elephant flows l Size of flows is huge l Duration of flows is long

Insight #2 of Science DMZ n Elephant flows l Size of flows is huge l Duration of flows is long Sufficient observation to infer HMMs for each flow Relatively “small” portion of traffic to analyze

Lightweight Detection System Design Incoming Traffic Symbol Table Online Detection Offline Training Detection Results HMMs Flow Rule Management Flow Rule Update

Offline Training Process Traffic Capturing Extracting Time Intervals Symbolization Inferring HMMs Symbol Table HMMs

Online Detection Process Traffic Capturing Packet Acquisition Extracting Time Intervals Compute Time Interval Symbolization Assign Symbol Table Inferring HMMs Count Probability of each Symbol HMMs Sufficient sampling Benign Chi-square test Matched Not matched Insufficient sampling Malicious

Evaluation n The processing time of each packet l Lightweight l IDS detection system instances l Expectation of the hybrid of two n False positive rate lightweight detection system

Evaluation Setup n Data Collection l Grid. FTP l Each (benign) vs. SCP (malicious) flow transfers 10 GB data l Generated through VMs on Cloud. Lab at Clemson n System Configuration l Implement l Bro lightweight detection system as IDS instances

Processing Time of Each Packet n T: expectation of processing time of each packet n L: processing time of lightweight detection system n I: processing time of IDS instances n : the ratio of traffic to analyze for filtering n : the ratio of malicious traffic n : false positive rate

Result Analysis Value Tested Variables Observed Value (Analysis ratio) 0. 1 L 0. 66 microsecond (Malicious ratio) 0. 5 I 44. 46 microseconds Parameters 0. 38 T 32. 7 microseconds

Result Analysis Value Tested Variables Observed Value (Analysis ratio) 0. 1 L 0. 66 microsecond (Malicious ratio) 0. 5 I 44. 46 microseconds Parameters 0. 38 32. 7 microseconds T Processing Time Per Packet (microsecond) 50 40 Saved 26% 30 20 10 0 LDS IDS instance Expectation

Result Analysis n Malicious ratio and FPR fixed n Analysis ratio and FPR fixed Parameters Value (Analysis ratio) 0. 1 (Malicious ratio) 0. 5 (FPR) 0. 38 n Analysis ratio and malicious ratio fixed Decreasing any of this parameters will further reduce the expectation of processing time!

Conclusion and Future Work n Conclusion l Present a new architecture for traffic monitoring in Science DMZ l Demonstrate efficiency of side-channel based traffic winnowing n Future work l l Evaluate on real-world Science DMZ data 4 More protocols 4 More use scenarios Investigate more side-channel features 4 Packet l size, time interval deviations, etc. Employ advanced machine learning techniques

Q&A Thank you!

Science DMZ Campus Router/Firewall Border Router Science DMZ Router Campus LAN Campus Router/Firewall WAN Border Router Science DMZ Router Campus LAN High performance Data Transfer Nodes

Threat Model Border Router WAN Compromised Data Transfer Node Malicious programs are installed and launched

Modeling Network Protocols n Network protocol modeling via Hidden Markov Models (HMMs) l C. Lu, et al. “A normalized statistical metric space for hidden markov models. ” l L. Yu, et al. “Inferring statistically significant hidden markov models. ” l X. Zhong, et al. “Side channel analysis of multiple pmu data in electric power systems. ”