Developing Video Applications on Xilinx FPGAs Xilinx Confidential

Developing Video Applications on Xilinx FPGAs Xilinx Confidential

Xilinx Design Flow Video Hardware Development 2 Xilinx / Avnet / Mathworks Video Seminar

Simulink Executable Spec • Purely algorithmic • Use abstract blocks • Define desired system response 3 Xilinx / Avnet / Mathworks Video Seminar

Executable Spec Demo 4 Xilinx / Avnet / Mathworks Video Seminar

Xilinx Design Flow Video Hardware Development 5 Xilinx / Avnet / Mathworks Video Seminar

Design a Hardware Architecture • Floating-point numbers • Defining basic elements of Hardware Architecture • Compare to Executable Spec 6 Xilinx / Avnet / Mathworks Video Seminar

Define Fixed-Point Quantization • Compare fixed-point error to golden source 7 Xilinx / Avnet / Mathworks Video Seminar

Redefine Dataflow for Hardware • Convert from frames to serial streaming • Consistent with CMOS video camera outputs 8 Xilinx / Avnet / Mathworks Video Seminar

Redefine design using the Xilinx DSP Blockset Gateway blocks define the FPGA boundary • Sys. Gen token block enables use of netlist generation scripts Xilinx DSP Blockset includes over 100 DSP building blocks that have been optimized for efficient results on Xilinx devices Define partition using Gateway. In / Gateway. Out blocks • 9 Xilinx / Avnet / Mathworks Video Seminar

Model Based Design Demo 10 Xilinx / Avnet / Mathworks Video Seminar

Xilinx Design Flow Video Hardware Development 11 Xilinx / Avnet / Mathworks Video Seminar

Video Hardware Verification Flow Golden Input Sequences & Test Cases 50+ Test Sequences Test Cases -Pedestrian Crossing -Abandon Object -Privacy Regions -Object Removal Reference Model Golden Test Vector Suite The Key is Fast Algorithm Confirmation • Simulink and the Video and Imaging blockset • Early System Architecture • Bit True, Not cycle accurate Micro-Architecture • IO Definitions • Automated TB Generation Design • API Definition Capture • Early FPGA Characterization Verification • HDL Simulation • Back Annotated HDL Sim • Sys. Gen Co. Sim Golden Test Vectors = ? • Sys. Gen-VSK Hw. Co. Sim • Sys. Gen-VSK-VFBC Hw. Co. Sim • Live Validation, Camera>VSK>Display 12 Xilinx / Avnet / Mathworks Video Seminar Validation

Accelerating Verification through Hardware • Hardware co-verification removes simulation bottleneck – Up to 1000 x simulation performance improvement – Automates FPGA and board setup process Simulation Time (Seconds) Design Softwar e HW Co. Sim Increas e Beamformer 113 2. 5 45 X OFDM BER Test 742 . 75 989 X DUC CFR 731 23 32 X Color Space Converter 277 4 69 x Video Scalar 10422 92 113 X 13 Xilinx / Avnet / Mathworks Video Seminar

Eliminating IO Bottlenecks using Hardware Frame Passing System Generator includes specials “Shared Memory Read / Write” blocks to allow large amounts of data to be efficiently passed to hardware from Simulink Boost HW co-sim performance by bundling input data samples together thus reducing simulation to hardware transactions. - Frame size =2880 14 Xilinx / Avnet / Mathworks Video Seminar

Simulation Runtime Improvements using Hardware Co-simulation Sim Time Abandon Object Design (seconds) Time / frame (seconds) Performance Improvement Original Simulink Abstract Model . 5 . 1 N/A With Sys. Gen block* 50 10 baseline Sys. Gen with HW co-sim no frames* 165 33 3 X slower Sys. Gen with HW co-sim with frames* 15 3 3 X With input from. mat file* 10 2 5 X With in/output. mat file* * Forto bit-true 15 4. 2. 8 12 X hardware accurate simulation models Xilinx / Avnet / Mathworks Video Seminar

Additional Data from Xilinx Video Development Team Frame s 800 x 600 1280 x 720 1920 x 1080 Non. Accelerated HW Accelerated Nonaccelerated HW Accelerated 1 1243 sec 18 sec 2238 sec 25 sec 5310 sec 37 sec 2 2579 sec 25 sec 4728 sec 35 sec NA 62 sec 5 NA 43 sec NA 65 sec NA 128 sec • Notes: – 100 Mbps Ethernet link, Effective rate ~5. 1 Mbps • ML 506 Virtex-5 SXT development platform – Payload = 2 * N_Frames * Frame_Size * 32 -bits – Load N frames of data, Process N Frames, Store N Frames 16 Xilinx / Avnet / Mathworks Video Seminar

Hardware Co-Simulation Demo 17 Xilinx / Avnet / Mathworks Video Seminar

Xilinx Design Flow Video Hardware Development 18 Xilinx / Avnet / Mathworks Video Seminar

Why Video Systems? • Video designs generally include embedded processing for: – Video system control and dataflow control – Table and memory updates – Low performance video processing • Xilinx embedded processors allow high- performance video systems on a single chip – Lower cost – Higher performance – Obsolescence proof 19 Xilinx / Avnet / Mathworks Video Seminar

The VSK Base System • • Embedded base system provided with the VSK Forms the framework from which video designs are created Includes one Micro. Blaze embedded processor Customized using Platform Studio 20 Xilinx / Avnet / Mathworks Video Seminar

System Generator to Embedded • System Generator automatically generates DSP accelerators for use with the Xilinx embedded development environment (XPS) – placed into embedded IP Catalog – Supports PLB or FSL bus – Supports async clocking – Includes driver files and documentation • XPS project can be 21 Xilinx / Avnet / Mathworks Video Seminar

Video Starter Kit Reference IP Use in Reference Designs Core Name DVI Passthrough Camera Frame Buffer DVI_IN and DVI_OUT Yes Yes Camera processing Yes DE_GEN Yes VIDEO_TO_VFBC Yes Video Frame Buffer Controller (VFBC) Yes • Provided as a library of “drag and drop” IP for use with the video base system • Provides abstraction to the video interface details – Includes SW driver files 22 Yes Reference IP included with the VSK Generated by Sys. Gen Xilinx / Avnet / Mathworks Video Seminar Platform Studio IP Catalog

Abstracting the Processor Interface • “Shared” registers, RAMs and FIFOs are used to create HW / SW abstraction – DSP design connects to a “to” or “from” memory – Memory maps and interface logic is added during RTL generation – Software drivers and documentation are created for easy programming 23 Xilinx / Avnet / Mathworks Video Seminar

System Design Integration Demo 24 Xilinx / Avnet / Mathworks Video Seminar

Video Example #1 - VFBC 25 Xilinx / Avnet / Mathworks Video Seminar

Getting Started with VSK Reference Designs Simplest Frame Buffer Data Transfer DVI Input DVI Output Frame Buffer Basic “real-time” video processing DVI Input Image Process ing DVI Output “Real-time” Frame Buffer Based Video Image Processing Camera DVI Frame Input 26 Process ing Buffer Xilinx / Avnet / Mathworks Video Seminar Output