Machine Learning Protocols in Automatic Myelin Segmentation Predrag

Machine Learning Protocols in Automatic Myelin Segmentation Predrag Janjić, MSc. Research Scientist, NIH Research Program in Psychiatric Diseases, RCCSIT@MANU pjanjic@manu. edu. mk

Myelin pathology – what do we do ? • Myelin pathology studies in our case are part of biological psychiatry research focused on pathology of schizophrenia, • conducted within structural studies of “Building Schizophrenia Research in Macedonia”project (PI’s Dwork and Rosoklija), run and coordinated by Neuropathology Lab at Psychiatric Institute of Columbia University in New York. • We seek changes in myelin expression at histological level, in post-mortem tissue of diagnosed and control subjects. • Apart of structural studies of myelin, the same autopsy samples are used for molecular biology studies of differential myelin expression and its regulation.

Who we are ? Andrew Dwork, MD, Ph. D Professor of Neuropathology, Psychiatric Institute, Columbia Univ. John Smiley, MD, Ph. D Research Scientist, Neurochemistry, Nathan S. Kline Institute, New York Gordana Petruševska, MD, Ph. D Professor of Pathology, Pathology Clinic, School of Medicine Ljupčo Kocarev, Ph. D Professor of Computer Science, Faculty of Informatics and Comp. Eng, RCCSIT at MANU Gorazd Rosoklija, MD, Ph. D Associate Professor of Psychiatry, Psychiatric Institute, Columbia Univ. Aleksandar Stankov, MD, Ph. D Forensic Pathologist, Institute of Forensic Medicine, Blagoja Dolgoski, MSc Electron Microscopist, Project Staff, School of Medicine Kristijan Petrovski, B. Sc Software Designer, Project staff at RCCSIT@MANU

Image segmentation

Myelin – what it is and how it looks like ? • Myelin – distinctive fatty structure which wraps and electrically insulates the axons. By preventing the rundown of electrical potential needed for the saltatory conduction, it supports efficient conductivity of neural impulses. Myelin accounts for the major part of brains white-matter. • Oligodendroglia is family of cells which members perform myelination of the axons in CNS. • Myelin is usually studied through expression of several dominant proteins, fats and enzymes (MBP, PLP 1, CNPase, specific glycolipids and several others) which builds up the sheath. • Myelination starts in the first year of life and is an intensive process until early adolescence. Myelin degradation is implicated in several diseases and medical conditions.

What questions regarding myelination do we ask ? • Can we see differences in myelin expression between diagnosed groups and controls ? • We measure the myelin content using g-ratio, which simply estimates relative contribution of myelin in fiber area of a single fiber. • Due to deformations on bending fibers we use values at low percentile of the myelin thickness distance map (over all measurements), e. g. 5 th or 10 th percentile

Segmenting myelin in white matter is challenging Zikopoulos & Barbas 2013, Front. Human Neurosci • Areas implicated in psychiatric diseases have very complex cytoand myeloarchitecture. • Fibers are branching in a wide spread of directions and angles. • There are no referent planes of cutting the tissue. • Bending fibers / bundles cut at small angles produce “distorted” cross-sections to be measured for both, myelin and axon thickness. • Tissue fixation adds several types of artifacts deformations.

Our histological samples • Samples of human prefrontal white-matter , H&E stained, have been visualized with TEM microscope, with 5. 000 x magnification. • For the development of the method we used 3 x 30 images, 2048 x 2048 pixels, 8 bit gray-scale. • Initial segmentation was attempted in Visiomorph® VIS image processing environment. • Segmentation was done in three classes, 1=Myelin, 2=Axon and 3=Background.

Machine learning in image processing q How to model and extract morphometric and/or other scale invariant features (SIFT) using computational structure ? o Perceptrons o Multi-Layer Perceptrons (MLP) o Deep Networks q Within image segmentation, ML algorithm / protocol should classify each pixel in one or more classes

Supervised vs. Unsupervised ML in Image Processing • Supervised learning requires fully annotated learning data • Unsupervised learning usually detects features from examples • Clustering methods in unsupervised discriminative learning put objects into different classes

Perceptrons and MLP’s Class-1 Class-2

Computational tools for segmentation • Trainable Weka (Image. J) • VIS suite

Deep Neural Networks in Image segmentation

Annotation of Image Sets for ML • Needed to produce “ground truth” for supervised ML • Most laborious step, which puts a limit on efficiency and productivity if to be done manually on large volume of images • “Gold-standard” for annotation of histologic samples – three independent annotations (done by microscopist, neuroanatomist and neuropathologist) in all images of the training set CORRECTION

Sampling strategies for input data • Pixel-based segmentation requires local-model of pixel intensity and neighboring relations (e. g. by convolution of a fragment). • Due to statistical nature of ML input data / fragments need Although we introduce a fragment, statistical sampling. segmentation the tool assigns a class • User defined features can, and should bias the sampling, if the central pixel, all other those are criticalonly fortodiscrimination. pixels just serve as a context

Specific sampling strategies for our input data • Optimal window size (33 x 33, 45 x 45, 65 x 65 pixel fragments) • Higher statistics of fragments along axon-myelin boundary • Equal representation of small and medium & large fibers according to area, • Certain minimal representation of “debris” objects • “Padding” at the edges for edge fibers • These and several other interventions are critical for learning efficiency and actual accuracy

Learning protocol with Convolutional DNN (Conv. Net) • A set of 30 full original images are pre-segmented using VIS Segmentation Tool, to get the Interim segmentation status, which reaches average pixel accuracy of 62. 4% (variable among the classes). • 2/3 of the set, 20 images are fragmented into 5 M – 8 M fragments, usually 45 x 45 pixels, which are introduced in pairs (Interim, Corrected) in fully supervised manner. • Convolution window size varies in narrow range (4 to 6 pixels) • There are typically more than one epoch (with same data) needed for the ML tool to saturate the values of interconnection weights • 5 of remaining images are used for verification which runs along the training, as a separate inputs set whose outputs are used for adjustments, but do not count for overall accuracy. • Fragments of 5 last images are used for testing when training is completed, i. e. the c. DNN tool is being run as a free classifier where it manifests segmentation capability

Initial findings • Average accuracy of 88% to 91% percent of pixel accuracy wa achieved (up from ~ 62. 4% with VIS), • Accuracy of myelin pixels was higher compared to axon and background pixels, • Parts of axonal substructure, pre-segmented as myelin were difficult to clean completely, • A certain volume of “debris” objects were remaining systematically, due to their structured noise nature (they often look like damaged fibers)

Extension of the DNN structure with pre-training layer • Instead of random initial values (of weights) in first input layer, we introduced an unsupervised pre-training as a single layer perceptron of specific type (Deep Belief Network), • This layer is trained with showing only input fragments from annotated images, which helps to achieve more realistic values for interconnection parameters of that layer once it goes back in DNN training protocols • This improved dramatically detection and removal of structured noise, i. e. spurious objects

Present quality of DNN segmentation protocol

How do we measure and average the g-ratio (why do removal of spurious objects matter) ?