Tensor Flow A System for LargeScale Machine Learning

Tensor. Flow: A System for Large-Scale Machine Learning Proc. of the 12 th USENIX Symposium of OSDI `16 Martin Abadi et al. Google Brain 오준오 Machine Learning & Pattern Analysis laboratory 18 th November 2019

| Index Related Works Deep Learning? Introduction Background & Motivation Design Principles Tensor. Flow Execution Model Extensibility Cases Studies Implementation Evaluation Conclusion

| Related Works

| Deep Learning? 위 작업의 반복

| Abstract Few Keywords: Tensor Large scale Heterogeneous environments Dataflow graph

| Introduction Tensor + Dataflow graph

| Tensor. Flow execution model 모든 머신러닝 알고리즘에 대해 Tensor. Flow의 dataflow graph를 나타냄

| Tensor. Flow execution model Single process client master process master Session run Execute subgraph worker process 1 worker GPU 0 GPU 1 GPU 2 worker process 3 GPU 0 GPU 1 GPU 2 … … 단일 서버와 분산 시스템의 구조

| Tensor. Flow execution model Device B B Y C W W RECV Key - Value Rendezvous key A Device A X Device A 분산 시스템에서의 실행 SEND A X

| Extensibility case studies Differentiation and Optimization Dist. Belief는 parameter server에 있는 구현체를 바꿔야 함 많은 사용자에게는 부담스러운 작업

| Extensibility case studies Fault tolerance 모든 parameter를 저장할 필요는 없음 주어진 input으로부터 다시 계산 가능하며, 대다수의 학습 알고리즘은 일관성을 요구하지 않음 Variable Save Restore Checkpoints(. ckpt)

| Implementation Core runtime 외에도 visualization dashboard(Tensorboard), quantization, inference tools for production 등 유용한 도구 제공

| Evaluation Single-machine benchmarks Six-core Intel Core i 7 -5930 K CPU + NVIDIA Titan X GPU Neon outperforms three by using hand-optimized convolutional kernels implemented in assembly language

| Evaluation Image Classification