Realtime data acquisition and processing applications using NIRIO

  • Slides: 24
Download presentation
Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs IMAGE

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs IMAGE ACQUISITION AND GPU PROCESSING APPLICATION USING IRIO TECHNOLOGY AND FLEXRIO DEVICES Julián Nieto, Mariano Ruiz, Sergio Esquembri, Guillermo de Arcas, Alberto de Gracia julian. nieto. [email protected] es Universidad Politécnica de Madrid 1

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Motivation

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Motivation q Develop high performance real-time applications for data acquisition (FPGA-based) and image processing (GPU) using PCIe with COTS solutions. q Integration in a distributed system managed by a control and instrumentation middleware as EPICS 2

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System architecture 3

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System architecture q Image acquisition standard: Camera. Link Full speed 850 MB/s v Proven standard q Availability of cameralink hardware simulator to replace the camera in the development/debugging phase. v 4

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System architecture: framegrabber NI Flex. RIO FPGA Module • Virtex-5 FPGA • 132 digital I/Ol ines • 512 MB of DRAM MXI Link • 4 lines used • Theorical bandwith 1 GB/S XILINX Virtex 5 FPGA NI Flex. RIO Adapter Module • Interchangeable I/O • Analog, digital, image • MDK for custom modules 5 PXI Platform • Syncronization • Clocking 7 triggers • Power/cooling • Data streaming

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs System architecture: GPU in a Industrial computer q Tesla / Quadro NVIDIA architecture selected v Use of NVIDIA RDMA feature Direct data movement from framegrabber to GPU memory v Use of Open Source Linux Device. Driver for NI-Flex. RIO Devices with this improvement included. Developed by National Instruments and UPM v This allows to move images from cameralink simulator to GPU memory up to 850 MB/s v 6

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Data

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Data flow in the system FPGA/GPU/CPU q In this solution the FPGA and the GPU can be consider as a whole. 7

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Starting

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Starting point to develop the software applications DAQ FPGA system GPU NVIDIA Distributed Control System Is it possible to accelerate the development time? 8

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Problems

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Problems and solutions Hardware/Software FPGA/Lab. VIEW/FPGA resource interface GPU Integration in EPICS Improvement Templates solving image acquisition and preprocessing Linux library, IRIO Libraries using CUDA model Development of a EPICS device support using a modularized approach with C++ classes q Solution: § IRIO software tools provided all these functionalities § Please see todays presentation Prof. Mariano Ruiz, RTA 1 session, 11: 20 9

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Using

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Using IRIO: standalone C and EPICS applications 10

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Standalone

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Standalone C application running in the CPU #. . . { Init FPGA & GPU resources } while(my. Status==IRIO_success && !Eacq){ my. Status |= irio_get. DMATto. GPUImage(&p_Drv. Pvt, IMGSIZE, 0, &gpu_ptr, &count, &status); if(my. Status==IRIO_success && count==IMGSIZE){ GPU_process(gpu_ptr, U 64 WORDS); irio_release. DMATto. GPUImage(&p_Drv. Pvt, ……. ); } else usleep(value); } #. . . { Free FPGA & GPU resources } 11

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Integration

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Integration with EPICS(I) q Why: § Because we want use EPICS environment for configuration and monitorization. q What do we need? § To develop a “customizable” device driver for EPICS. This is the connection between EPICS and hardware. § Nominal Device Support is a methodology developed by ITER/Cosylab to simplify this development. 12

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Integration

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Integration with EPICS(II) q Are we moving all images acquired and processed to EPICS? § No, that’s impossible. EPICS is used only for monitoring. Data archiving is implemented using other NDS elements and APIs providing this. q Our solution is a C++ library solving data acquisition in GPU processing q 13

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Classes

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Classes hierarchy in NDS Here we are defining all the basic things to initialize the device FPGA/GPU EPICS NDS C++ classes IOC Shell parse. Device. Params nds. Create. Devic e nds. IRIODevic e create. Structure parse. Channel. Groups. Params We are creating the data acquisition threads for the device NDS C++ classes channel = new nds. IRIOImage. Channel (name, type, NºChannel, ring. ID) nds. IRIOChannel. Group create. Channels register. Channel (channel) 14

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Classes

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Classes hierarchy in NDS nds. IRIODevice nds. IRIOChannel. Group on. Enter. Init init. Group • irio_init. Driver() • Channel. Group -> init. Group() • Irio_set. Up. DMAsto. GPU() • irio_config. CL() • GPU_Open() • GPU_Set. User. Buffer. U 64() nds. IRIOImage. Channel on. Enter. Processing Image. Channel. Publishing • epics. Ring. Bytes. Get() • do. Callbacks. Int 8 Array() on. Enter. Processing on. Enter. On • Irio_set. FPGAStart() close. Driver Image. DMAGPU_threa d on. Leave. Processing • Irio_get. DMAto. GPUImage() • process. Image_GPU() • epics. Ring. Bytes. Put() • irio_release. DMAto. GPUImag e() NDS provides default states machine You only need to customize this for your specific application 15

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Example

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Example of JPEG codec compression cuda. JPEGDevice parse. Channel. Group. Para ms cuda. JPEGChannel. Group process. Image_GPU() • epics. Ring. Bytes. Put() • *channel. Group = new cuda. JPEGChannel. Group() • channel. Group -> create. Channels() 16 cuda. JPEGImage. Channel. Publishing • epics. Ring. Bytes. Get() • do. Callbacks. Int 8 Array()

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs One

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs One simple example Frame-grabber acquiring 2300 fps (256 x 256) q Image inversion and JPEG Compression in GPU: 0, 29 ms q Image displayed in CSS (raw image and compressed image) using EPICS. q Configuration, etc using EPICS PVs. q 17

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Conclusions

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Conclusions q Optimized solution for the use of data movement from DAQ and GPU memory using RDMA NVIDIA technology. q Solution provided with: § NI-RIO Linux Device Driver (Open source). FPGA-based (Flex. RIO) data acquisition devices from National Instruents § A C++ library developed using Nominal Device Support approach for EPICS integration. • In use in ITER for image diagnostics q FPGA frame-grabber implemented in the FPGA adding the possibility of implement preprocessing. Implementation through Lab. VIEW for FPGA. 18

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Conclusions

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Conclusions q Software tools available as GPL V 2 § IRIO: UPM § NI RIO Linux: National Instruments § NDS: Cosylab q Simplified methodology for the development of realtime advanced data acquisition and image processing applications. q Similar applications demanding parallel data acquisition and massive processing 19

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs IMAGE

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs IMAGE ACQUISITION AND GPU PROCESSING APPLICATION USING IRIO TECHNOLOGY AND FLEXRIO DEVICES Universidad Politécnica de Madrid More detailed info. Short course RT 2016 : Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Please contact with Julián Nieto (julian. nieto. [email protected] es) 20

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs q.

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs q. BACKUP 21

22 Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs

22 Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Approach

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs Approach 2: EPICS NDS application Using GPUS with IRIO-NDS Uso de clases Herencia Adqusición CL Imágenes Procesado JPEG 23

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs More

Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs More information q IRIO Technology: developing applications for Advanced DAQ systems using FPGAs § Prof. Mariano Ruiz, RTA 1 session, [email protected]: 20 q Short curse RT 2016 teached June 6 TH : § Real-time data acquisition and processing applications using NIRIO FPGAs-based technology and NVIDIA GPUs 24