ISMBECCB 2017 Chromatin Accessibility Prediction via Convolutional Long

ISMB/ECCB 2017 Chromatin Accessibility Prediction via Convolutional Long Short-Term Memory Networks with k-mer Embedding Xu Min, Wanwen Zeng, Ning Chen, Ting Chen*, Rui Jiang* Presenter: Xu Min Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China minxueric@gmail. com

Contents • Background • Method • Results 2017/7/24 Xu Min, Tsinghua University 1

Contents • Background • Method • Results 2017/7/24 Xu Min, Tsinghua University 2

Background • What is chromatin accessibility? (Wang et al. , 2016) • Biological experiments such as DNase-seq, FAIRE-seq, and ATAC-seq, etc. • Expensive & time consuming. 2017/7/24 Xu Min, Tsinghua University 3

Previous work • Computational methods mainly fall into two classes: 1. kmer-based methods – e. g. kmer-SVM (Lee et al. , 2011), gkm-SVM (Ghandi et al. , 2014) – Pros: feature sets for arbitrary-length sequences – Cons: capture only local motif patterns 2. deep learning-based methods (CNN-based) – e. g. Deep. Bind (Alipanahi et al. , 2015), Deep. SEA(Zhou and Troyanskaya, 2015), Basset (Kelley et al. , 2016), Deep. Enhancer (Min et al. , 2016) – Pros: detect motifs automatically, superior performance – Cons: one-hot encoding, require fix-length input 2017/7/24 Xu Min, Tsinghua University 4

Our idea • Regard DNA sequences as sentences. Sentence classification (Kim, 2014) • Word embedding: Skip-gram (Mikolov et al. , 2013) and Glo. Ve (Pennington et al. , 2014) 2017/7/24 Xu Min, Tsinghua University 5

Contents • Background • Method • Results 2017/7/24 Xu Min, Tsinghua University 6

Our approach • Our approach: Convolutional long short-term memory networks with k-mer embedding. • Combine two classes of methods: – We regard one DNA sequence as a kmer sequence, and train kmer vector by Glo. Ve. – We then use convolutional LSTM network for supervised learning to classify input sequences. 2017/7/24 Xu Min, Tsinghua University 7

Method • Feature learning in three stages: • Classification loss function: 2017/7/24 Xu Min, Tsinghua University 8

k-mer embedding with Glo. Ve • Learning embedding representations mainly from the cooccurrence statistics information of k-mers. • Glo. Ve model’s cost function: • Embedding k-mers by Glo. Ve unsupervised results: 2017/7/24 Xu Min, Tsinghua University 9

Bidirectional LSTM • Produce a fixed-length output features. • Learn long-range relationships of DNA sequences. 2017/7/24 Xu Min, Tsinghua University 10

Novelties • We fuse the informative k-mer features into a deep neural network by embedding k-mers into a low dimensional vector space. • We are able to handle variable-length DNA sequences as input and capture long-distance dependencies thanks to LSTM units. 2017/7/24 Xu Min, Tsinghua University 11

Contents • Background • Method • Results 2017/7/24 Xu Min, Tsinghua University 12

Experiments setup • Dataset: ENCODE DNase-seq experiments for 6 cell lines. • Train: validation: test sets 0. 85: 0. 05: 0. 10 2017/7/24 Xu Min, Tsinghua University 13

Experiments setup • Unsupervised training of k-mer embedding: – Kmer length k=6, Splitting stride s=2 – Glo. Ve: window size=15, embedding dimension=100 • Supervised deep learning architecture: – Keras + Theano – RMSprop: learning rate=0. 001, batch size=3000 – Early stopping: maximum iterations=60, patience=5 2017/7/24 Xu Min, Tsinghua University 14

Model Evaluation 2017/7/24 Xu Min, Tsinghua University 15

Model Evaluation 2017/7/24 Xu Min, Tsinghua University 16

Visualization of k-mer embedding 2017/7/24 Xu Min, Tsinghua University 17

Visualization of k-mer embedding 2017/7/24 Xu Min, Tsinghua University 18

Efficacy of k-mer embedding 2017/7/24 Xu Min, Tsinghua University 19

Efficacy of convolution and BLSTM 2017/7/24 Xu Min, Tsinghua University 20

Sensitivity analysis 2017/7/24 Xu Min, Tsinghua University 21

Conclusion • Main contributions summarized as below: – We innovatively introduce an effective embedding representation of input DNA sequences using the unsupervised learning algorithm Glo. Ve in the deep learning framework. – We are capable of handling variable-length sequences as input and capturing complex long-range dependencies on them by exploiting the BLSTM network. – We prove our model produces state-of-the-art performance in sequence classification tasks, compared to other recent methods including gkm. SVM and Deep. SEA. 2017/7/24 Xu Min, Tsinghua University 22

Reference • Alipanahi, B. et al. (2015) Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nat. Biotechnol. , 33(8), 831– 838. • Ghandi, M. et al. (2014) Enhanced regulatory sequence prediction using gapped k-mer features. PLo. S Comput. Biol. , 10, e 1003711. • Kelley, D. R. et al. (2016) Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks. Genome Res. , 26(7), 990– 999. • Kim, Y. (2014). Convolutional neural networks for sentence classification. In: Conference on Empirical Methods on Natural Language Processing (EMNLP), Association for Computational Linguistics (ACL), pp. 1746– 1751. • Lee, D. et al. (2011) Discriminative prediction of mammalian enhancers from dna sequence. Genome Res. , 2167– 2180. • Mikolov, T. et al. (2013). Distributed representations of words and phrases and their compositionality. In: Burges, C. J. C. et al. (eds) Advances in Neural Information Processing Systems, NIPS. Curran Associates, NY 12571. pp. 3111– 3119. • Min, X. et al. (2016). Deep. Enhancer predicting enhancers by convolutional neural networks. In: IEEE International Conference on Bioinformatics and Biomedicine, IEEE, pp. 637– 644. • Pennington, J. et al. (2014). Glo. Ve: global vectors for word representation. In: EMNLP, volume 14, p. 1532– 43. • Wang, Y. et al. (2016) Modeling the causal regulatory network by integrating chromatin accessibility and transcriptome data. Natl. Sci. Rev. , 3(2), 240– 251. • Zhou, J. , and Troyanskaya, O. G. (2015) Predicting effects of noncoding variants with deep learning-based sequence model. Nat. Methods, 12, 931– 934. 2017/7/24 Xu Min, Tsinghua University 23

Thank you! Q&A Travel Fellowship Generously Supported by Hit. SEQ COSI: High Throughput Sequencing Algorithms & Applications 2017/7/24 Xu Min, Tsinghua University 24