Welcome Hong Yan Dept of Electronic Eng City

Welcome Hong Yan Dept of Electronic Eng, City University of Hong Kong School of Electrical & Information Eng, University of Sydney

Outline of Presentation * Clustering and biclustering * Data partition in terms of objects and features * Bicluster models in multidimensional space * Biclustering algorithms * Applications to microarray data classification * Cancer data analysis * Three-color microarray data analysis * Discussions

Members of Our Group People who have contributed to this work: Xiangchao Gan Qinghua Huang Haizhou Li Alan W C Liew Hongya Zhao

1 D Data Clustering

2 D Data Clustering Clusters (classes): 10 characters Objects: 100 character images Features: pixel values How to cluster the data: check the similarity / distance in terms of features

Classification based on Clustering Objects Features Objects Cluster 3 Cluster 4 Cluster 2 Cluster 1 Features Clustering in row direction Clustering in column only direction only Clustering in either row or column but not both directions Usually full coverage of data matrix Usually no overlap between clusters Contiguous indexing always possible

Bicluster 2 Objects Bicluster 4 Bicluster 1 Bicluster 3 Classification based on Biclustering A cluster: subset of objects, all features, OR subset of features, all objects A bicluster: subset of objects (rows) AND subset of features (columns) Features Clustering in both row and column directions Usually not full coverage of data matrix Possible overlap between biclusters Contiguous indexing mostly unlikely

Contiguous Row and Column Indexing Bicluster 3 Bicluster 1 Bicluster 2 Bicluster 4 Contiguous row and column indexing by reordering * Possible for one bicluster at a time * Usually impossible for all biclusters at the same time if overlapping allowed Hard to visualize all biclusters simultaneously

Pattern Similarity and Coherent Values Random data? No correlation? Try points 1, 3, 5 and 8 for red and blue lines. Similar pattern and coherent values!

Distance Measure Distance measure not appropriate for biclustering Problem 1: it includes junk features Problem 2: it does not reveal pattern similarity Red + Cyan lines: Small distance, but different patterns No coherence Red + Blue lines Large distance, but similar patterns Coherent values

Applications of Biclustering Applications - Find a subset of genes to be co-expressed under the same subset of conditions - Find a subset of voters who choose the same subset of candidates - Find a subset of currencies with same changes for a subset of market conditions - Find a subset of customers who buy the same subset of products - Find a subset of documents that use the same subset of common words Related algorithms - Projective (projected) clustering, subspace clustering - Feature selection (PCA, ICA, transformation and combination) Key problems in biclustering - Choose a subset of features and a subset of samples at the same

Microarray Experiments (Image from IEEE Potentials, Feb/Mar 02)

Color Microarray Image Expressed in reference cell Expressed in reference and tumor cell Expressed in tumor cell Not expressed

Genes and Conditions Gene 1 Condition N Condition 2 Condition 1 Gene M Unordered data in condition direction unless they represent time series.

Microarray Time Series Gene 1 t. Nt 1 t 0 Gene M 1 Ordered data in time direction.

Microarray Data Matrix N Conditions M Genes

Time Series Data Representation M Genes t 0 t 1 N time points t. N-1 Ordered data

Computational Complexity of Biclustering Left: original data matrix, size MXN Middle: rows and columns permuted Total number of permutations = (M!)(N!) Stigma: * NP-hard * Inherently intractable

Different Types of Biclusters Constant Additive Constant row Multiplicative Constant column Linear

Mean Squared Residue Score Cheng and Church’s Algorithm Column average I: subset of genes J: subset of conditions I X J: bicluster Row average Pattern average

The Plaid Model

Other Existing Methods Examples of existing algorithms: Iterative row and column clustering combination Divide and conquer, greedy iterative search Exhaustive bicluster enumeration Distribution parameter identification Examples of existing methods: Coupled two-way clustering (CTWC) Iterative signature algorithm (ISA) Statistical-algorithmic method for bicluster analysis (SAMBA) Spectral biclustering Machine learning based methods Information-theoretic co-clustering

Our Recent Work on Biclustering Our key strategies (1) Line / hyperplane detection (2) To find coherent data values and filter junk values (3) (2) Parameter estimation (4) To model different types of biclusters Advantages of our approach (1) Unified formulation of different biclusters (2) Biclustering transformed to image analysis (3) and parameter estimation problems (4) (3) Can separate different types of

Geometrical Interpretations Constant z=x=a Additive z=x+b Constant row z = x Multiplicative z=ax Constant column x = a, z = b General linear z=ax+b

Hyperplane Intersections Intersection of x = a and y = Intersection of x = a and z = c b No restriction on y No restriction on z Intersection of x = a, y = b and z = c All three variables have restrictions and form a coherent pattern.

Hough Transform

Lines in the Hough Space y b b = - a xk + yk (x 2, y 2) (x 1, y 1) (xk, yk) x bop t aop a t b = - a x 1 + y 1 y=ax+b b = - a x 2 + y 2

Parameter Quantization bmax bmi n amax ami n

Hyperplane in Object Space Hyperplane equation Condition variables Hyperplane coefficients Parameter value range

Hyperplane in Parameter Space Original hyperplane equation in object space Hyperplane coefficients Hyperplane variables Normalization Hyperplane equation in parameter space Only k out of M variables needed

The Fast Hough Transform Root (11, 01) 00) (1, (0, (10, 00) (1, 0) (10, 01) (1, 1) 0 ) (0, 1 (10, 00) ) (1, 0 ) 1 ) (11, 11) (11, 00) k-tree representation (10, 01) (0, 0) (0, 1) Developed by: Li, Levin and Le Master Test for hyperplane and hypercube intersection

Recursive Testing criterion Center for node at position b Distance at the top level Distance at level l Testing at level l

Analysis of the FHT based Method Advantages Can detect all hyperplanes at the same time No explicit cost function used Local search not needed Very fast if the number of conditions is small Tolerant to noise using coarse to fine testing Disadvantage Large memory requirement if too many conditions Solutions Slipt and merge the conditions Search in column-pair space and combine

Examples of Simulated Biclusters Original data Random matrix? Row and column reordered data One constant row bicluster One additive bicluster

Hyperplane Model of a Constant Row Bicluster

Hyperplane Model of an Additive Bicluster

Column-pair Space Junk features Useful features Junk features

How to Detect Lines or Hyperplanes Line detection using Hough transform (HT) features: Irrelevant No coherence Randomly distributed Discarded Useful features: Coherent values Line/hyperplane structure Discovered by the HT

How to Detect Hyperplane Intersections We vote you!!

Models for Different Types of Biclusters General form of lines: Curves can also be modelled if biologically meaningful.

Different Types of Voting Patterns Direct voting; a variant form of the HT; more efficient.

Dynamic Range of Scaling Factor Angle measure has much smaller dynamic range. It’s a better quantity to use.

Simulated Biclusters of Coherent Values Constant Column Constant Row Additive Multiplicative

Column-Pair Voting Result Example 1 xj 2 Xj 1 = Xj 2 = a Xj 1 = Xj 2 + b

Column-Pair Voting Result Example 2 xj 1 xj 2 Xj 1 = a, Xj 2 = b Xj 1 = a Xj 2

Bicluster Voting Result Example Original Data Constant Column Constant Row Additive Multiplicati ve

Noisy Bicluster Models Consider Distributions of Model Parameters Parameter Estimation Separation of Different Biclusters

Performance for Noisy Data CC: Cheng and Church algorithm ISA: Iterative signature algorithm biclustering OPSM: Order-preserving sub-matrix Binmax: Divide and conquer MSBE: Maximum similarity GBC: Geometric biclustering

Performance for Overlapping Biclusters CC: Cheng and Church algorithm ISA: Iterative signature algorithm biclustering Binmax: Divide and conquer MSBE: Maximum similarity

Lymphoma Data Classification 298 genes 156 gene s Human Lymphoma Data (Alizadeh et al) 4026 genes, 96 conditions Data values: -749 to 642 349 gene s

Breast Cancer Data Classification 830 genes 34 conditi ons Breast Cancer Data (van ‘t Verr et al) 20000 genes, 5000 used 98 conditions 641 genes 42 conditi ons

Human Organ Data Analysis

GO Analysis of Human Organ Biclusters

Three-Color Microarray Data

Hexa. Mplot Slant axis: perfect drug II, V: drug works, not enough III, VI: drug works, too strong I, IV: bad drug, side effect B=R (Proposed by H Zhao, R Wong, K Fang and P Yue)

Studying Effect of Rg 1 on Hcy Cy 5 (Red): normal HUVECs Cy 3 (Green): Hcy-related HUVECs Alex 549 (Blue): Rg 1 treated Hcy-relate HUVECs Rg 1: extract of ginsenosides in ginseng Hcy: homocysteine-treated human umbilical vein endothelial cells (HUVECs)

Patterns in the Hough Space Looking for coherent patterns Up-regulated genes

Drug Therapeutic Effect Assessment

Foreign Currency Exchange Data Analysis * Objects: currencies AUD (Australian Dollar), CAD (Canadian Dollar), CHF (Swiss Franc), DKK (Danish Kroner), EUR (European Euros), GBP (British Pounds), JPY (Japanese Yen), KRW (South Korean Won), NOK (Norwegian Kroner), NZD (New Zealand Dollar), SEK (Swedish Krona), SGD (Singapore Dollar), THB (Thai Baht), TWD (Taiwan Dollar), and ZAR (South African Rand) * Features: time 137 months points

Currency Data

Bicluster of European Currencies 5 European currencies have consistent movement patterns SEK has large fluctuations

Bicluster of Asian Currencies 3 Asian currencies have consistent movement patterns THB has large fluctuations

Future Work * Removing noise and combining small biclusters * Bicluster validity * Biological meanings of biclusters * Feature normalization and invariance * Pattern recognition * Other applications, e. g. DNA sequence analysis Stock price analysis

Noise Removal Bicluster 1 Bicluster 3 Bicluster 2

Feature Normalization Similar patterns but scaled along time axis

Summary * Efficient biclustering methods developed * Useful for DNA microarray data analysis * Applications to disease identification * Applications to drug therapeutic effect assessment * Noise removal and bicluster validity needed * Feature normalization and invariance needed * Many other applications

Common Language Molecular Biology Architectur e Circuits Informatio n Messages Networks Signals Systems Electronic Engineeri ng

Building a Bridge

Thank You Contact e-mail: h. yan@cityu. edu. hk for