FPGAs in the real world Programmable Solutions Group

FPGAs in the real world Programmable Solutions Group Rebecca Nevin

AGENDA • Features of FPGAs • Real World Applications • FPGAs vs. GPUs for AI • How to Learn More PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 2

Setting the Stage

The Problem: Flood of Data By 2020 The average internet user will generate ~1. 5 GB of traffic per day Smart hospitals will be generating over 3 TB per day Self driving cars willbe be generating over Self driving cars will over GB per day…each 44, 000 TB per day… A connected plane will be generating over 40 TB per day A connected factory will be generating over 1 PB per day All numbers are approximated http: //www. cisco. com/c/en/us/solutions/service-provider/vni-network-traffic-forecast/infographic. html http: //www. cisco. com/c/en/us/solutions/collateral/service-provider/global-cloud-index-gci/Cloud_Index_White_Paper. html https: //datafloq. com/read/self-driving-cars-create-2 -petabytes-data-annually/172 http: //www. cisco. com/c/en/us/solutions/collateral/service-provider/global-cloud-index-gci/Cloud_Index_White_Paper. html ~10100 radar KB ~10100 sonar KB ~50 per secon d per second

The Solution: Smarter Compute High-performance computing (HPC) is the use of parallel processing for running advanced application programs efficiently, reliably and quickly. The term applies especially to systems that function above a teraflop or 1012 floating-point operations per second (definitions may vary) Assuming 3 cycle per multiplication operation on a 3 GHz processor. A single threaded processor can produce 1 new multiplication product every 1 billionth of a second. A 2012 Image. Net classification algorithm* takes a 256 x 256 pixel image and classifies it against 1000 categories that the image might map to. This (unoptimized) algorithm takes 12. 2 trillion multiplies! With a single multiply product every 1 billionth of a second, it would take 12, 200 seconds to categorize (3 hrs, 24 minutes) at one multiply every one billionth of a second. *https: //vast. cs. ucla. edu/sites/default/files/publications/CNN_ICANN 14. pdf 5

Hardware Acceleration § Offloading tasks from the CPU to other more appropriate hardware § We’ve been doing this for decades with discrete graphics, audio, DSP, etc. § Programmable logic has been around since the 80’s, and is finally the popular kid Video: Accelerating Mandelbrot 6

FEATURES OF FPGAs

What is an FPGA • Flexible, multi-functional reprogrammable silicon • Custom hardware functionality • Bare-metal speed and reliability • Custom parallelism PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 8

Benefits of FPGA Technology • Performance • Time to market • Cost • Reliability • Energy Efficiency • Long-Term Maintenance PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 9

THE Balancing Act CPU GPU FPGA ASIC & ASSP Heterogeneous Peak Performance Moderate High Very Highest Very High Power Consumption High Very Lowest Moderate Flexibility Highest Moderate Very High Lowest Very High Cost Moderate High Very Highest* Very High* Parallelism Very Low Very High Custom In reality, one architecture cannot solve all the world’s compute problems. PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 10

Real-World Applications

5 G Wireless Aerospace Networking FPGA FOCUS MARKETS Cloud Computing PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Smart Cities Autonomous Driving 12

Transforming Data Centers CPU GPU ASSP Artificial Intelligence Big Data Analytics FGP As (Hadoop, SPARK, SQL, No. SQL) ASIC FPGA Video Transcoding Network functions virtualization Storage Acceleration Security and DPI (Deep Packet Inspection) PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 13

CASE STUDY: Microsoft 125%29% 8 X Gain in Throughput PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential DECREASE Increase in speed in Latency With 15% less power 14

Accelerating the Critical tasks of autonomous driving Sensor Fusion FGP As AI/Machine Learning Functional Safety 5 g connectivity PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 15

Case Study: Embedded Systems Video: How Intel FPGAs Enable the Industrial Internet of Things PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 16

Case Study: Video and Vision Real-Time Analytics § Simultaneous motion detection, facial recognition, and object detection § Multiple input feeds Flexible Sensor Interfaces Signal Processing Video Compression § Adapt to changes in proprietary interfaces without changing the rest of the design § Pick and choose video processing functionality using IP cores § Accelerate pre-processing of high-res videos § Integrate CODECs with other processing functions on a single FPGA PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 17

What is AI? Artificial Intelligence Sense, learn, reason, act, and adapt to the real world without explicit programming Perceptual Understanding Detect patterns in audio or visual data Data Analytics Build a representation, query, or model that enables descriptive, interactive, or predictive analysis over any amount of diverse data Machine Learning Computational methods that use learning algorithms to build a model from data (in supervised, unsupervised, semi-supervised, or reinforcement mode) Deep Learning Algorithms inspired by neural networks with multiple layers of neurons that learn successively complex representations Convolutional Neural Networks (CNN) DL topology particularly effective at image classification PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 18

AI is Transforming Industries CONS HEalth FINAN RETAI GOVE ENER TRAN INDUS CE L UMER SPOR TRIAL RNME GY Support Smart Algorithmic Assistants Enhanced Defense Smart Grid Automated Cars Factory Trading T NT Diagnostics Experience Automation Chatbots Search Drug Discovery Personalization Patient Care Augmented Reality Research Sensory Aids Robots Fraud Detection Research Personal Finance Risk Mitigation PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Data Insights Conservation Marketing Merchandising Loyalty Safety & Security Resident Engagement Supply Chain Smarter Cities Operational Improvement Oil & Gas Exploration Automated Trucking Predictive maintenance Aerospace Shipping Precision Agriculture Field Automation Security 19

Intro to DEEP LEarning Labelled Training Data Flower Model Training Bus New Data PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Inference Prediction Model Flower 20

BENEFITS OF Intel FPGAS f. OR AI HARDWARE PERFORMANC WORKLOAD Power Efficiency E FLEXIBILITY Deployment Flexibility FLEXIBILITY Reduced total cost of ownership Offloaded or in-line processing Customizable precision and data types Speed I/O Types Future Algorithms Real-time decision making Direct interface to data source Adaptable to architectures of the future Throughput Power envelope Multi-functionality Do more with less Only use as much as you need AI and more, all on one chip PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Precision 21

Intel® FPGA : application specific performance The right performance and features for the right application Management, Sensors and edge devices Vision systems, and purpose-built, application-specific hardware Cloud, datacenter, and HPC Scalable and efficient computing performance Datacenter / CSP Acceleration 5 G Wireless Infrastructure Datacenter Network Communications Networking Machine Vision Board Management Edge Compute I/O Expansion Embedded Vision Military / Defense ADAS Robotics Infotainment Automobile sensors, traffic sensors Copyright © 2017, Intel Corporation. All rights reserved.

High-Level Design (HLD) Portfolio FRONT-END Software Programmer PERSONA Algorithm Designer Intel® FPGA SDK for Open. CL™ Generated RTL DSP Builder for Intel® FPGAs Generated RTL Quartus® Prime Embedded Designer Intel® HLS Compiler Hardware Designer HDL Code, Qsys (schematic) PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Generated RTL 23

How to Choose FRONT-END • What are you trying to achieve? • Which specs (speed, power, cost, time to market) are most important? • How many units will you need? • Where will it be deployed? • How often to you expect your design to change? • What is the expertise of your engineering team? • Has someone already built a solution that’s “good enough”? PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential CPU GPU FPGA ASIC & ASSP Heterogeneo us Peak Performance Moderate High Very Highest Very High Power Consumption High Very Lowest Moderate Flexibility Highest Moderate Very High Lowest Very High Cost Moderate High Very Highest* Very High* Parallelism Very Low Very High Custom 24

The Next 5 -10 Years FRONT-END • Time to Market/Ease-of-Use • Fractal Synthesis • High-level Synthesis (HLS) • Improving compilation times • One. API, many processors • Efficiencies in neural networks • Hardware Architectures • Applications • Heterogeneous and hybrid computing • 5 G and beyond • Chiplets & CHIPS • Compression • Arithmetic design • Whatever you think of next! • Security, Security • Algorithms PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 25

READY TO LEARN MORE? Intel® FPGA Academic Program Online tutorials, labs, curriculum, software and hardware. Intel AI Academy Videos and classes on AI, Machine Learning and Deep Learning Online Training Deepen your expertise with Intel FPGA training courses. Design Examples Get started with Intel FPGA products with ready-to-use design examples Community Forum Get your questions answered by Intel® FPGA technical experts Intel FPGA You. Tube Short videos on tools and technology Click the images to be taken to our website PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 26

Thank You! fpgauniversity. intel. com PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Rebecca Nevin Program Manager Intel® FPGA Academic Program rebecca. l. nevin@intel. com 27

HOW FPGAs WORK

Back to the BASICSS A B Z 0 0 0 1 0 A Z 0 1 1 0 0 1 0 1 1 0 0 0 1 1 1 1 0 0 0 1 1 1 1 0 0 Inverter Z = ~A OR =A|B PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential AND Z=A&B 2: 1 MUX Z = (~S & A) | (S & B) 1 1 30

LUT: THE FOUNDATION PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 31

LOGIC ARRAY (Building) BLOCKS x 10 PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 32

Building the Array LABs Row interconnect Column interconnect PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential Segmented interconnects 33

How its Programmed Row/Column Interconnect Junction Programming info stored in a external nonvolatile device • Active: programmed automatically at power-on • Passive: Intelligent host (CPU) controls programming PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 34

Modern FPGAs Source: https: //www. altera. com/products/fpga/cyclone-series/cyclone-v/features. html PROGRAMMABLE SOLUTIONS Programmable Solutions Group GROUP | Intel® Confidential 35