DATE 2008 Design Automation Test in Europe DATE

DATE 2008 Design, Automation & Test in Europe (DATE) 2008 An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System A. Das , S. Misra, S. Joshi*, J. Zambreno+, G. Memik, and A. Choudhary Electrical Engineering and Computer Science Department *Indian Institute of Technology, Kharagpur, + Iowa State University

Overview Principal Component Analysis PCA Framework Results DATE 2008 Contributions q Novel architecture for Principal Component Analysis (PCA) used in Network Intrusion Detection q Parallel implementation of PCA on an FPGA platform q Achieving a throughput of 24. 72 Gbps with up to 99. 9% accuracy and as low as 1. 95% false alarm rate 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 2

Overview Principal Component Analysis PCA Framework Results DATE 2008 Overview: Network Intrusions q Network Attacks / Intrusions § § § Malicious Code (virus, worms, Trojan horses, malware) Approximately $70 B economic losses a year (and increasing) Example attacks: • • 10/2/2020 Denial of Service (Do. S) Port scanning A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary SYN Flood by exploiting TCP 3 -way handshake An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 3

Overview Principal Component Analysis PCA Framework Results DATE 2008 Overview: Network Intrusions q Network Attacks / Intrusions § § § Malicious Code (virus, worms, Trojan horses, malware) Approximately $70 B economic losses a year (and increasing) Example attacks: • • 10/2/2020 Denial of Service (Do. S) Port scanning A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary Smurf attack An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 4

DATE 2008 Overview: NIDS q Network Intrusion Detection System § § q Suspicious Activity Monitoring both inbound and outbound Identify attacks and alert the System Administrator/User NIDS techniques: § Signature Detection • § less false alarms, new attacks undetected (e. g. SNORT) Anomaly Detection • q Overview Principal Component Analysis PCA Framework Results more false alarms, detection of new attacks Popular Anomaly Detection Methods: § § 10/2/2020 Data Mining, Genetic algorithms, Statistical analysis Performance Bottlenecks: S/W methods cannot match Gigabit Ethernet rates A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 5

DATE 2008 NIDS: Challenges q Problems with Signature IDS § q Can only detect known intrusion types Problems with Anomaly IDS § § 10/2/2020 Overview Principal Component Analysis PCA Framework Results Need to have “sufficient” training data that covers as much variation of the normal behavior as possible Too much dependence over training data set False positives Slow due to high dimensional network data A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 6

DATE 2008 NIDS: Challenges q Problems with Signature IDS § q Overview Principal Component Analysis PCA Framework Results Can only detect known intrusion types Problems with Anomaly IDS § § Need to have “sufficient” training data that covers as much variation of the normal behavior as possible Too much dependence over training data set False positives Slow due to high dimensional network data Offline Phase Labeled Training Data Learning Normal Behavior Preprocessing Deviation Online Phase Live Network Stream 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary Attack Detection An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 7

DATE 2008 Overview Principal Component Analysis PCA Framework Results Overall NIDS Framework 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 8

DATE 2008 Outline q q Overview of Network Intrusions Detection Principal Component Analysis (PCA) § § q § PCA architecture FPGA implementation Results § § § q Distance Calculation Methodology PCA Framework § q Overview Principal Component Analysis PCA Framework Results FPGA performance Accuracy H/W Speedup Conclusions 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 9

Overview Principal Component Analysis PCA Framework Results DATE 2008 Principal Component Analysis (PCA) q Express variation in multivariate data in uncorrelated variables q Input data dimension consists of p correlated variables x 1, x 2, …xp q PCA looks for a transformation of the xi into p new variables yi that are uncorrelated y 1 f 1(x 1, x 2, …xp) y 2 f 2 (x 1, x 2, …xp) Eigenvalues (λ ) ……… Eigenvectors (e ) yp fp(x 1, x 2, …xp) i i 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 10

Overview Principal Component Analysis PCA Framework Results DATE 2008 Original Variable B PCA: An Example PC 2 PC 1 Original Variable A PC 1 is the direction of maximum variance (major) q PC 2 uncorrelated (orthogonal) to PC 1 (minor) q 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 11

Overview Principal Component Analysis PCA Framework Results DATE 2008 PCA: Properties q q Principal components are arranged in descending order indicating the amount of variation it encompasses Keep the q most important ones and discard the others Some deviation may not follow the same correlation model § May want to keep the r least important components Calculate distance in the new set of axes § Outlier if distance far from normal 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 12

Overview Principal Component Analysis PCA Framework Results DATE 2008 PCA Major principal components Minor principal components 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary y 1 y 2. . yq. . yp-r+1. . yp An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 13

Overview Principal Component Analysis PCA Framework Results DATE 2008 PCA: Steps involved q Distance Calculation: Map the input data into suitable axes; calculate the distance using principal components § § Euclidian Distance (equal weight for each dimension) Mahalanobis Distance (uses correlation matrix) q Outlier Detection: Distance exceeding a predefined threshold § 10/2/2020 t. M and tm with Major and Minor components, respectively A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 14

Overview Principal Component Analysis PCA Framework Results DATE 2008 PCA Framework: Phases Offline Phase Online Phase 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 15

DATE 2008 Overview Principal Component Analysis PCA Framework Results PCA Framework: Phases Offline Phase: q Step 1: Mean vector of the sample data (training data) created q Step 2: Correlation Matrix R created q Step 3: Eigen-analysis performed on R; extraction of <ei, λi> pairs q Step 4: Pairs are sorted according to λi to determine the major and minor principal components q Step 5: Thresholds are calculated 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 16

DATE 2008 Overview Principal Component Analysis PCA Framework Results PCA Framework: Phases Online Phase: q Step 1: Online data (test data) are mapped to the eigenspace of q & r principal components q Step 2: Maj. C and Min. C scores are computed in parallel q Step 3: Threshold comparison is performed 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 17

Overview Principal Component Analysis PCA Framework Results DATE 2008 PCA: Advantages q Reduces the dimensionality of network data with little or no information loss § q Complete breakdown into offline and online phase § q Eigen-analysis and sorting performed in offline phase Major and minor components chosen define two thresholds § q Analyses different network TCP parameters (features) Better coverage Faster compared to software implementation § 10/2/2020 Hardware level parallelism to compute PC score A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 18

DATE 2008 Overview Principal Component Analysis PCA Framework Results Principal Component Score Pipeline for FPGA 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 19

Overview Principal Component Analysis PCA Framework Results DATE 2008 FPGA Implementation Design synthesis using VHDL q Place and route using Xilinx ISE 8. 1 q Target Platform Xilinx XC 2 VP 30 q # of pipeline stages = F(p, q, r) q Training and testing dataset : q § KDD Cup 1999 100, 000 -125, 000 training data samples q Up to 12 of 32 principal components used q Speedup limited by I/O bandwidth of FPGA board q 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 20

DATE 2008 Outline q q Overview of Network Intrusions Detection Principal Component Analysis (PCA) § § q § PCA architecture FPGA implementation Results § § § q Distance Calculation Methodology PCA Framework § q Overview Principal Component Analysis PCA Framework Results FPGA performance Accuracy H/W Speedup Conclusions 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 21

Overview Principal Component Analysis PCA Framework Results DATE 2008 Results: FPGA Performance q # fields (p) = 32 q Maximum throughput = 24. 72 Gbps #q=4 #q=8 Throughput (Gbps) 25 24 23. 5 23 22. 5 22 21. 5 21 0 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary 1 #r 2 4 An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 22

DATE 2008 Overview Principal Component Analysis PCA Framework Results: PCA Accuracy Detection and False Alarm Rates vs. q 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 23

DATE 2008 Overview Principal Component Analysis PCA Framework Results PCA hardware Speedup q Compared with sequential software code q System: AMD Opteron 2. 4 GHZ with 2 GB memory 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 24

Overview Principal Component Analysis PCA Framework Results DATE 2008 Conclusions q q q 10/2/2020 High Ethernet throughput demands efficient anomaly detection in networks High dimensional network data becomes a bottleneck for anomaly detection Hardware implementation of NIDS based on Principal Component Analysis can be effective Parallelism is exploited using reconfigurable hardware (FPGAs) Attack detection up to 99% with false alarm rates as low as 1. 95% Significant speedup compared to software A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 25

DATE 2008 Acknowledgements This work was supported by NSF grants NSF-ITR CCR-0325207, CNS-0406341, CNS-0551639, IIS-0536994, CCR-0325207, by Air Force Office of Scientific Research (AFOSR) award FA 9550 -06 -10152 and Do. E CAREER Award DE-FG 02 -05 ER 25691 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 26

DATE 2008 Thank You ! Questions? Contact Info: Electrical Engineering and Computer Science Department Northwestern University 2145 Sheridan Road Evanston, IL 60208 Phone: (847) 467 -4610 Fax: (847) 467 -4144 Email: ada 829@eecs. northwestern. edu Web: http: //www. ece. northwestern. edu/~ada 829 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 27

DATE 2008 References q q q q 10/2/2020 [1] M. E. Attig and J. Lockwood. A framework for rule processing in reconfigurable network systems. In IEEE Symposium on Field-Programmable Custom Computing Machines, (FCCM), Napa, CA, Apr. 2005. [2] Z. K. Baker and V. K. Prasanna. Time and Area Efficient Pattern Matching on FPGAs. In The Twelfth Annual ACM International Symposium on Field-Programmable Gate Arrays (FPGA ’ 04), 2004 [3] Z. K. Baker and V. K. Prasanna. Efficient Hardware Data Mining with the Apriori Algorithm on FPGAs. In Proceedings of the Thirteenth Annual IEEE Sym. on Field Programmable Custom Computing Machines 2005 [4] S. Dharmapurikar, P. Krishnamurthy, T. Sproull, and J. W. Lockwood. Deep packet inspection using parallel bloom filters. In Symp. on High Performance Interconnects, August 2003. [5] M. Fleury, B. Self, and A. C. Downton. A fine-grained parallel pipelined karhunen-loeve transform. In 17 th International Parallel and Distributed Processing Symposium, Nice, France, April 2003. [6] J. D. Jobson. Applied Multivariate Data Analysis, Volume II: Categorical and Multivariate Methods. Springer-Verlag, NY, 1992. [7] I. T. Jolliffe. Principal Component Analysis. Springer-Verlag, NY, 2002. A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 28

DATE 2008 References q q q q 10/2/2020 [8] Jungsuk Song and Hiroki Takakura and yasuo Okabe. A proposal of new benchmark data to evaluate mining algorithms for intrusion detection. In 23 rd Asia Pacific Advanced Networking Meeting, 2007. [9] KDD Cup 1999 Data. http: //kdd. ics. uci. edu/databases/kddcup 99. html, August 1999. [10] Nicholas Athanasiades and Randal Abler and John Levine and Henry Owen and George Riley. Intrusion detection testing and benchmarking methodologies. In IEEE International Information Assurance Workshop, 2003. [11] R. Sidhu and V. Prasanna. Fast regular expression matching FPGAs. In IEEE Symposium on Field-Programmable Custom Computing Machines, April 2001. [12] D. V. Schuehler, J. Moscola, and J. W. Lockwood. Architecture for a hardware-based, tcp/ip content-processing system. In IEEE Micro, January 2004. [13] M. -L. Shyu, S. -C. Chen, K. Sarinnapakorn, and L. Chang. A novel anomaly detection scheme based on principal component classifier. In IEEE Foundations and New Directions of Data Mining Workshop, pages 172– 179, November 2003. [14] H. Song and J. W. Lockwood. Efficient packet classification for network intrusion detection using fpga. In Intl. Symp. On Field-Programmable Gate Arrays, February 2005. A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 29

DATE 2008 Backups 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 30

DATE 2008 Overview Principal Component Analysis PCA Framework Results NIDS FPGA Architecture Feature Extraction 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary Principal Component Analysis An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 31

DATE 2008 Feature Extraction Module HF(1) Sketch table 1 32 Src IP Dst IP Src port Dst port Flags HF(2) 32 16 16 Hash Control Block : : 6 HF(H) Architecture A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary : : : : Feature Sketch 10/2/2020 : Sketch table 2 Estimate Block Estimate d value Sketch table H An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 32

DATE 2008 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 33

DATE 2008 Overview Principal Component Analysis PCA Framework Results: PCA Accuracy Detection and False alarm Rates vs. q 10/2/2020 A. Das, S. Misra, S. Joshi, J. Zambreno, G. Memik and A. Choudhary An Efficient FPGA Implementation of Principal Component Analysis based Network Intrusion Detection System 34