Acumos Generate Microservices for ONNX model Bruno Lozach

Acumos™ Generate Micro-services for ONNX model Bruno Lozach – Orange 07/21/2020

Acumos™ : Approach for generating microservice for ONNX models Goal : Exporting a Model from Py. Torch to ONNX and Running it as Acumos micro service My approach : - ONNX Runtime Why : ONNX Runtime is a performance-focused engine for ONNX models, which works efficiently across multiple platforms and hardware (Windows, Linux, and Mac and on both CPUs and GPUs). ONNX Runtime has proved to considerably increase performance over multiple models - Super. Resolution model as proposed by Londhe and Guy Jacobson 2 Interne Orange Priyadarshini https: //wiki. acumos. org/pages/viewpage. action? page. Id=20548067

Acumos™ : Approach for generating microservice for ONNX models Super-resolution is a way of increasing the resolution of images, videos and is widely used in image processing or video editing. For this example, we will use a small super-resolution model. This model uses the efficient sub-pixel convolution layer described in “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network” - Shi et al For increasing the resolution of an image by an upscale factor. The model expects the Y component of the YCb. Cr of an image as an input, and outputs the upscaled Y component in super resolution. 3 Interne Orange

Acumos™ Approach for generating microservice for ONNX models Efficient Sub-Pixel Convolutional Neural Network 4 Interne Orange

Acumos™ Approach for generating microservice for ONNX models SR Model use torch nn conv 2 D and Re. LU for CNN https: //github. com/pytorch/examples/blob/master/super_resolution/model. py 5 Interne Orange

Acumos™ Approach for generating microservice for ONNX models Training : Ordinarily, you would now train this model however, for this example, we will instead download some pre-trained weights. # Load pretrained model weights model_url = 'https: //s 3. amazonaws. com/pytorch/test_data/export/superres_epoch 100 -44 c 6958 e. pth' Notes : This model was not trained fully for good accuracy and is used here for demonstration purposes only. The input size will be fixed in the exported ONNX graph for all the input’s dimensions [224, 224]. 6 Interne Orange

Acumos™ Approache for generating microservice for ONNX models Using Our Example Model On Acumos Platform 1. Onboarding 2. Downloading (retrieve model’s docker image) 3. Deploying to Docker 4. Communicating with the Model Microservice 7 Interne Orange

Acumos™ Approach for generating microservice for ONNX models Onboarding Our Example Model # Some imports. . def run. Super. Resolution. Onnx. Model(image. Input. Data: bytes )-> bytes: """ This method take in inputs bytes representing an image and return bytes representing the 672 x 672 "Upscaled and Enhanced" image. We first resize the image to fit the size of the model’s input (224 x 224). Then we split the image into its Y, Cb, and Cr components. These components represent a greyscale image (Y), and the blue-difference (Cb) and red-difference (Cr) chroma components. The Y component being more sensitive to the human eye, we are interested in this component which we will be transforming. After extracting the Y component, we convert it to a tensor which will be the input of our model. """ …. . ort_session = backend. prepare(onnx_model) ort_outs = ort_session. run(img_y) … return image. Output. Data. . # prepare Acumos Dump and Push session req_map=dict(PIL='pillow', onnx='onnx', onnxruntime='onnxruntime ') requirements = Requirements(reqs=['PIL'], req_map=req_map) model = Model(run. Super. Resolution. Onnx. Model=run. Super. Resolution. Onnx. Model) # push ~/my-model-image on Acumos plateform session. push(model, 'Super. Resolution. Onnx. Model', requirements=requirements, options=opts) 8 Interne Orange

Acumos™ Approach for generating microservice for ONNX models Running the model on an image using ONNX Runtime For this example, we will use a famous cat image used widely which looks like below 9 Interne Orange

Acumos™ Approach for generating microservice for ONNX models Communicating with the Model Microservice (Easy to use) import super. Resolution. Onnx_model_pb 2 as pb import requests rest. URL = http: //localhost: 3330/model/methods/run. Super. Resolution. Onnx. Model headers = {'Content-Type': 'application/vnd. google. protobuf', 'accept': 'application/vnd. google. protobuf' } def Super. Resolution. Onnx. Model(image. Input. Data: bytes)-> bytes: """ This method take in inputs bytes representing an image and return bytes representing the "Upscaled and Enhanced" image. """ input = pb. Runsuperresolutiononnxmodel. In() buffer = io. Bytes. IO(image. Input. Data) input. image. Input. Data = buffer. getvalue() print("*** Call ONNX Runtime Prediction ***") r = requests. post(rest. URL, headers=headers, data=input. Serialize. To. String()) output = pb. Runsuperresolutiononnxmodel. Out() output. Parse. From. String(r. content) 10 return output. value Interne Orange

Acumos™ Approach for generating microservice for ONNX models Communicating with the Model Microservice (Easy to use) # Open and prepare cat image file in order it can be processed with our Super. Resolution. Onnx. Model method input. Image. File. Name = "cat. jpg“ input. Image. File = io. open(input. Image. File. Name, "rb", buffering = 0) image. Input. Data = input. Image. File. read() # bytes type # Call Super. Resolution. Onnx. Model method with our Cat image. Output. Data = Super. Resolution. Onnx. Model(image. Input. Data) # bytes type # Save the processed image in new fileoutput. Image File. Name = 'try_super_resolution_' + input. Image. File. Name output. Image. File = io. open(output. Image. File. Name, "wb", buffering = 0) output. Image. File. write(image. Output. Data) 11 Interne Orange