Research Interests 1 My Research Interests Distributed algorithms

Research Interests 1

My Research Interests Distributed algorithms g Distributed shared memory systems g Distributed computations over wireless networks g Distributed optimization/machine learning 2

My Research Interests Distributed algorithms g Distributed shared memory systems g Distributed computations over wireless networks g Distributed optimization/machine learning 3

Deep Neural Networks neuron layer 1 x 1 W 111 a 1 1 a 2 x 2 2 a 3 x 3 W 132 3 a 4

input parameters layer 1 x 1 W 111 1 x 2 2 x 3 3 W 132 output 0. 8 dog 0. 1 cat 0. 09 ship 0. 01 car

Deep Neural Networks layer 1 x 1 a 1 1 a 2 x 2 2 a 3 x 3 W 132 3 a 4

x 2 x 3 W 131 W 132 3 s(X 2 W 131+X 3 W 132+b 13)

Rectifier Linear Unit x 2 x 3 s(z) z W 131 W 132 3 s(X 2 W 131+X 3 W 132+b 13) z

How to train your dragon g Given a machine structure g Parameters are the only free variables network Choose parameters that maximize accuracy 9

How to train your network g Given a machine structure g Parameters are the only free variables Choose parameters to maximize accuracy Optimize a suitably defined cost function h(w) to find the right parameter vector w

So far … h(w) Training Optimize cost function h(w) Machine parameters w 11

Distributed Machine Learning g Data is distributed across different agents i. Mobile users i. Hospitals i. Competing vendors

Distributed Machine Learning g Data is distributed across different agents Agent 1 Agent 2 Agent 3 Agent 4 i. Mobile users i. Hospitals i. Competing vendors

Distributed Machine Learning g Data is distributed across different agents Collaborate to learn

Distributed Machine Learning g Data is distributed across different agents h 1(w) Collaborate to learn h 2(w) Training Optimize cost function Σ hi(w) i h 3(w) h (w) Machine parameters w

Distributed Machine Learning g Research problems – Privacy-preserving distributed optimization – Robustness to adversarial agents