TENSORFLOW MODELS ACCELERATED FOR JETSON John Welsh July

Tensor. Flow models on Jetson AGENDA Create an object detection dataset Train an object

TENSORFLOW MODELS ON JETSON Image Classification Models Object Detection Models Datasets + Training Scripts

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model

TENSORFLOW TRAINING cat dog cat Likelihood “cat” Likelihood “dog” 8

TENSORRT INTEGRATION Tensor. RT Optimized Graph Original Tensor. Flow Graph A A C B

IMAGE CLASSIFICATION Classification Runtime on Jetson TX 2 140 124 120 100 74. 3

OBJECT DETECTION Detection Runtime on Jetson TX 2 132 140 120 100 72. 9

FORMAT MODEL Remove/Replace Unsupported Operations 6 Force Device Placement Feature Extractor GPU Box Decoder

Slides: 24

Download presentation

TENSORFLOW MODELS ACCELERATED FOR JETSON John Welsh, July 27 2018

Tensor. Flow models on Jetson AGENDA Create an object detection dataset Train an object detection model Optimize the model with NVIDIA Tensor. RT 2

TENSORFLOW MODELS ON JETSON Image Classification Models Object Detection Models Datasets + Training Scripts cat dog cat Golden Retriever Miniature Poodle Toy Poodle https: //github. com/tensorflow/models https: //github. com/NVIDIA-Jetson/tf_trt_models 3

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 4

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 5

SCREENCAST – DATASET CREATION 6

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 7

TENSORFLOW TRAINING cat dog cat Likelihood “cat” Likelihood “dog” 8

SCREENCAST – TRAINING 9

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 10

NVIDIA TENSORRT 11

TENSORRT INTEGRATION Tensor. RT Optimized Graph Original Tensor. Flow Graph A A C B D E Tensor. RT Optimization TRT Op E 12

IMAGE CLASSIFICATION Classification Runtime on Jetson TX 2 140 124 120 100 74. 3 80 Golden Retriever 55. 1 60 40 Miniature Poodle 20 Toy Poodle 0 22. 9 3. 72 7. 99 Mobilenet V 1 128 28. 9 20. 7 12. 5 7. 36 Inception V 1 Inception V 3 TF-TRT (FP 16) Res. Net V 1 50 Res. Net V 1 152 TF (FP 32) 13

OBJECT DETECTION Detection Runtime on Jetson TX 2 132 140 120 100 72. 9 80 60 54. 4 50. 5 40 dog 20 0 Mobile. Net V 1 SSD TF-TRT (FP 16) dog dog Inception V 2 SSD TF (FP 32) 14

OBJECT DETECTION Detection Runtime on Jetson TX 2 132 140 120 100 72. 9 80 60 54. 4 50. 5 40 dog 20 dog dog 0 Mobile. Net V 1 SSD TF-TRT (FP 16) Inception V 2 SSD TF (FP 32) 15

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 16

FORMAT MODEL Remove/Replace Unsupported Operations 6 Force Device Placement Feature Extractor GPU Box Decoder GPU Non-max Suppression CPU Subtract Re. LU 6 Re. LU Subtract 17

SCREENCAST – FORMAT MODEL 18

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 19

TENSORRT INTEGRATION Tensor. RT Optimized Graph Original Tensor. Flow Graph A A C B D E Tensor. RT Optimization TRT Op E 20

SCREENCAST – TENSORRT OPTIMIZATION 21

TENSORFLOW MODELS ON JETSON Desktop / Server Create Dataset Train Model Jetson Format Model Optimize with Tensor. RT https: //github. com/NVIDIA-Jetson/tf_trt_models 22

SCREENCAST – VIDEO DEMO 23