Machine Learning for Motor Imagery Decoding ECE228 Project

Problem Description • • • The brain, which is responsible for nearly all functional

Motor Imagery Decoding • Prior to the onset of movement, there is a steady

Literature Survey Shakeel et al, CMMM, 2015

Dataset Description • Dataset consists of 60 hours of EEG recordings across 13 participants

Data Performance in paper differed between subjects Kaya et al, Sci Data 5, 2018

Pre-Processing Techniques: Common Spatial Patterns • Common spatial patterns (CSP) is widely used in

Classification Models • • Input data domain: timeseries data. Approach: Use NN models that

Classification Results of LSTM Network Topology Pre-processing None LSTM 100 LSTM 50 -Dense 2

Classification Results of LSTM - Dropout Network Topology Pre-processing None LSTM 100 LSTM 50

Pre-Processing Techniques: Wavelet-BSS Raw Data ● 1. 2. 3. Wavelet-BSS is a layered preprocessing

Wavelet Transformation Approximate blocks of EEG data with non-sinusoidal, non-fourier transforms since EEG noise

BSS via Independent Component Analysis Blind Source Separation can be done with either PCA

Clean data in Wavelet-ICA space Within the observational space of data projected onto Independent

Training LSTM on Wavelet-BSS Reduced Overfitting but did not improve performance.

SVM Classification Model ● ● SVM. Commonly used, simple, and effective 2 cases (left

Further Items to be Completed • • • Attempting to reduce LSTM overfitting by

References • Kaya, M. , Binli, M. , Ozbay, E. et al. A large

Slides: 20

Download presentation

Machine Learning for Motor Imagery Decoding ECE-228 Project Zhimin Liang, Daniel Valencia, Nicholas Wong

Problem Description • • • The brain, which is responsible for nearly all functional behavior of the human body, can be impeded by spinal-cord injury and other neurodegenerative diseases. Brain-computer interfaces (BCI) are devices that can record and stimulate different parts of the brain and may provide alternative pathways for neural signals. Recording/Sensing may be invasive/non-invasive. Electroencephalograms (EEG) is an example of a non -invasive recording method. A brain-computer interface setup for controlling a prosthetic limb

Motor Imagery Decoding • Prior to the onset of movement, there is a steady decrease in the EEG amplitude measured over the motor cortex, known as MRCP. • The MRCP is observed in both real and imagined movement. • Motor imagery (MI) decoding involves classifying the MRCP waveforms for different movements. The MRCP of a healthy subject for real and imagined movement. Shakeel et al, CMMM, 2015

Literature Survey Shakeel et al, CMMM, 2015

Dataset Description • Dataset consists of 60 hours of EEG recordings across 13 participants • 8 male, 5 female • Four MI paradigms: Five finger • movement (5 F), Classic (CLA), Freeform (FREEFORM), Hands Legs Tongue (Ha. LT) Recordings contain 22 channels of EEG data, and markers are given that denote the movement type for each time step. X 3 https: //commons. wikimedia. org/wiki/File: 21_electrodes_of_In ternational_10 -20_system_for_EEG. svg

Data Performance in paper differed between subjects Kaya et al, Sci Data 5, 2018

Pre-Processing Techniques: Common Spatial Patterns • Common spatial patterns (CSP) is widely used in EEG analysis. • Involves constructing spatial features that will optimize the variance between time-series data of different classes. • For MI decoding, this increases the difference between EEG waveforms of different classes. • Using sum of squared difference: • • SSD without CSP: 16. 9 e+3 SSD with CSP: 31. 7 e+6 Raw Data w/ CSP Wang et Al, Clinical Neurophysiology, 1999

Classification Models • • Input data domain: timeseries data. Approach: Use NN models that store temporal information within the model. Recurrent Neural Networks (RNNs) compute their hidden and output states based on the current input and previous hidden states. Dense layer has as many units as there are MI classes (one-hot encoding). LSTM 100 EEG Data t=1 LSTM 50 t=2 … Dense N MI Prediction t=�� The LSTM RNN for decoding MI tasks

Classification Results of LSTM

Classification Results of LSTM Network Topology Pre-processing None LSTM 100 LSTM 50 -Dense 2 CSP None CSP LSTM 100 LSTM 50 -Dense 5 None CSP MI Paradigm CLA 5 F Ha. LT Training Accuracy (%) Testing Accuracy (%) 97. 470 57. 276 100. 000 84. 554 93. 476 54. 292 94. 896 74. 959 97. 813 57. 653 95. 776 75. 388

Classification Results of LSTM - Dropout Network Topology Pre-processing None LSTM 100 LSTM 50 -Dense 2 CSP None CSP LSTM 100 LSTM 50 -Dense 5 None CSP MI Paradigm CLA 5 F Ha. LT Training Accuracy (%) Testing Accuracy (%) 96. 641 58. 103 99. 301 87. 503 91. 408 52. 064 90. 499 74. 947 97. 152 57. 188 92. 177 75. 035

Pre-Processing Techniques: Wavelet-BSS Raw Data ● 1. 2. 3. Wavelet-BSS is a layered preprocessing technique combining wavelet transformation, ICA and standard linear denoising techniques. Perform Wavelet Transformation (meyer or morlet wavelet) Perform Blind Source Separation via ICA or PCA and project data into component ‘observational’ space. Clean data in the ‘observational’ space via linear techniques or unsupervised machine learning (Kalman filters, SVM, Auto Regression, Outlier Detection, …) 4. Inverse BSS 5. Inverse wavelet potential glitch or artifact Cleaned Data

Wavelet Transformation Approximate blocks of EEG data with non-sinusoidal, non-fourier transforms since EEG noise is often not time dependent. EEG data may be discretized, or kept continuous. Discretization sometimes improves classification of EEG data. Jiang et al. 2019 Common wavelets used in EEG preprocessing are Discrete Wavelet Transforms using Morlet or Meyer waves mother waves. Preference for non orthogonal wavelets to compliment ICA.

BSS via Independent Component Analysis Blind Source Separation can be done with either PCA or ICA to identify prominent features and reduce dimensionality. ICA is preferred because it separates components based on least amount of gaussianity rather than orthogonality giving features of most signal information. Clean in Independent Component Space Transform cleaned data back to source space

Clean data in Wavelet-ICA space Within the observational space of data projected onto Independent Component Space, we can apply conventional cleaning techniques Auto Regression Kalman Filter Outlier Detection Detrending Data over all Channels SVM to identify noise and remove Transform back into source space after cleaned with Inverse ICA/PCA and Inverse Wavelet Transformation

Training LSTM on Wavelet-BSS Reduced Overfitting but did not improve performance.

SVM Classification Model ● ● SVM. Commonly used, simple, and effective 2 cases (left hand, right hand) Training: 99% Test: 90% ● 3 cases (left hand, no signal, right hand) ● Everything classified as a left hand signal https: //commons. wikimedia. org/wiki/File: Kernel_Machine. svg

Further Items to be Completed • • • Attempting to reduce LSTM overfitting by incorporating cleaning data using Wavelet-BSS and Dropout together Additional non-linear cleaning techniques within ICA space Testing the LSTM on the FREEFORM paradigm Training the LSTM on data spanning multiple subjects Data augmentation Online processing

References • Kaya, M. , Binli, M. , Ozbay, E. et al. A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces. Sci Data 5, 180211 (2018). https: //doi. org/10. 1038/sdata. 2018. 211 • Shakeel, Aqsa, et al. “A Review of Techniques for Detection of Movement Intention Using Movement-Related Cortical Potentials. ” Computational and Mathematical Methods in Medicine, vol. 2015, pp. 1– 13. , doi: 10. 1155/2015/346217. • Wang, Yunhua, Patrick Berg, and Michael Scherg. "Common spatial subspace decomposition applied to analysis of brain responses under multiple task conditions: a simulation study. " Clinical Neurophysiology 110. 4 (1999): 604 -614. • Jiang, X. , Bian, G. , Tian, Z. Removal of Artifacts from EEG signal: A Review. MDPI (2019)

Thanks for watching