Standard Binaries for FPGAs e Blocks Frank Vahid

"Standard Binaries for FPGAs" & "e. Blocks" Frank Vahid Professor Department of Computer Science and Engineering University of California, Riverside Associate Director, Center for Embedded Computer Systems, UC Irvine Work supported by the National Science Foundation, the Semiconductor Research Corporation, Xilinx, Intel, and Freescale Contributing Students: FPGAs: Roman Lysecky (Ph. D 2005, Asst. Prof. at U. Arizona), Greg Stitt (Ph. D 2007, Assistant Prof. at U. Florida), Ann Gordon-Ross (Ph. D. 2007, Asst. Prof. at U. Florida), David Sheldon (4 th yr Ph. D), Scott Sirowy (3 rd yr Ph. D) e. Blocks: Susan Lysecky (Ph. D 2006, Asst. Prof. at U. Arizona), Ryan Mannion (3 rd yr Ph. D), Shawn Nemetebakshi (MS 2005), plus numerous undergraduate students Frank Vahid, UC Riverside

Software. . . Microprocessor instructions FPGA circuits n Software is no longer just "instructions" n The elephant of software has a (new) tail – FPGA circuits Frank Vahid, UC Riverside 2

FPGAs: The Quietly Arriving New Software FPGAs implement circuits as software – bits downloaded into memory Circuit a 2 x 2 switch matrix b a b F G 4 x 2 Memory a 1 00 1 1 a 0 01 1 0 1 1 10 0 0 LUT 11 d 0 F 1 0 a b 0 1 11 01 x SM 00 11 11. . . 01 FPGA SM SM 00 01 01. . . LUT y 11 SM LUT 10 SM 11 SM G n FPGA n Multi-billion dollar growing industry (Field-programmable gate array) – thousands of LUTs and switch matrices, plus flip-flops, multipliers, RAMs, etc. n Tools automatically compile circuits into bits that program FPGA n Increasingly found in embedded system products – medical devices, basestations, set-top boxes, etc. Frank Vahid, UC Riverside 3

Why Circuits (Sometimes) Beat Instructions C Code for FIR Filter for (i=0; ii << 128; i++) y[i] += += c[i] ** x[i]. . . Circuit for FIR Filter * * * + + + Processor n 1000’s of instructions n Several thousand cycles FPGA Processor n ~ 7 cycles n Speedup > 100 x In general, FPGA better due to circuit's concurrency, from bit-level to task level Frank Vahid, UC Riverside 4

Extensive Studies over Past Decade n Large speedups on many important applications n Numerous dedicated conferences (FPGA, FCCM, FPL, . . . ) Frank Vahid, UC Riverside 5

New FPGA Compilers Start from Common Programming Languages C, C++, Java Binary HDL Binary Synthesis FPGA Profiling Compiler Binary Microprocessor Frank Vahid, UC Riverside Bitstream Binary n Commercial products appearing in recent years n Good news FPGA 6

Problem: Best Temporal/Spatial Algorithms Differ Algorithm quicksort( array, left, right){ } if right > left: pivot= array[left] newpivot = partition(array, left, right, pivot) quicksort(array, left, newpivot -1) quicksort(array, newpivot + 1, right) Bitonic Sorting Network Frank Vahid, UC Riverside Platform Processor FPGA 7

Bigger Problem: Algorithms Matter Even More Quicksort Bitonic Sort Processor n FPGA For portability, need algorithms that are efficient on both n "Compromise programming" Frank Vahid, UC Riverside 8

New FPGA Compilers Start from Common Programming Langauges C, C++, Java Binary n FPGA Profiling Compiler n Binary Microprocessor Bitstream Binary BUT – Standard tools/binaries important for "ecosystem" Countless ideas failed for not respecting the ecosystem FPGA Languages Standard binary Tools Frank Vahid, UC Riverside Architectures 9

One Solution: Hide the FPGA n SW Binary Standard Profiling Compiler Today's microprocessors use dynamic translation (e. g. , x 86) n Binary n Translator RISC architecture VLIW architecture For FPGAs: Transparently translate standard microprocessor binaries to FPGAs n Translator Proc. Frank Vahid, UC Riverside FPGA Different architectures hidden Warp Processing n Developed at UCR 2002 present 10

Warp Processing Background: Basic Idea 1 Initially, software binary loaded into instruction memory Profiler I Mem µP D$ FPGA Frank Vahid, UC Riverside Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 On-chip CAD 11

Warp Processing Background: Basic Idea 2 Microprocessor executes instructions in software binary Profiler I Mem µP D$ FPGA Frank Vahid, UC Riverside Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 On-chip CAD 12

Warp Processing Background: Basic Idea 3 Profiler monitors instructions and detects critical regions in binary Profiler beq beq beq add add add µP I Mem D$ FPGA Frank Vahid, UC Riverside On-chip CAD Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 Critical Loop Detected 13

Warp Processing Background: Basic Idea 4 On-chip CAD reads in critical region Profiler I Mem µP D$ FPGA Frank Vahid, UC Riverside Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 On-chip CAD 14

Warp Processing Background: Basic Idea 5 On-chip CAD converts critical region into control data flow graph (CDFG) Profiler I Mem µP D$ FPGA Dynamic Part. On-chip CAD Module (DPM) Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 reg 3 : = 0 reg 4 : = 0 loop: reg 4 : = reg 4 + mem[ reg 2 + (reg 3 << 1)] reg 3 : = reg 3 + 1 if (reg 3 < 10) goto loop ret reg 4 Frank Vahid, UC Riverside 15

Warp Processing Background: Basic Idea 6 On-chip CAD synthesizes decompiled CDFG to a custom (parallel) circuit Profiler I Mem µP D$ FPGA Dynamic Part. On-chip CAD Module (DPM) Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 + + + reg 3 : = 0 + : = 0+ reg 4 + . . . loop: +reg 4 : = reg 4++ mem[ reg 2 + (reg 3 << 1)] reg 3 : = reg 3 + 1 if (reg 3. . loop +< 10). goto ret reg 4 Frank Vahid, UC Riverside + . . . 16

Warp Processing Background: Basic Idea 7 On-chip CAD maps circuit onto FPGA Profiler I Mem µP D$ FPGA Dynamic Part. On-chip CAD Module (DPM) Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: Shl reg 1, reg 3, 1 Add reg 5, reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 + reg 3 : = 0 + : = 0+ + reg 4 SM SM SM. . . loop: reg 4+ + mem[ + CLB + +reg 4 : =CLB + reg 2 + (reg 3 << 1)] reg 3 : = reg 3 + 1 SM + if. SM (reg 3 goto. . loop +< 10). SM + + ret reg 4 Frank Vahid, UC Riverside + . . . 17

Warp Processing Background: Basic Idea 8 On-chip CAD replaces instructions in binary to use hardware, causing performance and energy to “warp” by an order of magnitude or more Profiler I Mem µP D$ FPGA Dynamic Part. On-chip CAD Module (DPM) Feasible for repeating or longrunning applications Frank Vahid, UC Riverside Software Binary Mov reg 3, 0 Mov reg 4, 0 loop: // instructions Shl reg 1, reg 3, that 1 interact FPGA Add reg 5, with reg 2, reg 1 Ld reg 6, 0(reg 5) Add reg 4, reg 6 Add reg 3, 1 Beq reg 3, 10, -5 Ret reg 4 + Software-only “Warped” reg 3 : = 0 + : = 0+ + reg 4 SM SM SM. . . loop: reg 4+ + mem[ + CLB + +reg 4 : =CLB + reg 2 + (reg 3 << 1)] reg 3 : = reg 3 + 1 SM + if. SM (reg 3 goto. . loop +< 10). SM + + ret reg 4 + . . . 18

Recent Warp Results on Multi-Threaded Benchmarks n After translation (may take minutes), huge speedups n n Even compared to 64 -microprocessor system [ Stitt/Vahid CODES/ISSS 2007] Translation results remembered for later execution Frank Vahid, UC Riverside 19

FPGAs as Software: Challenges and Opportunity n Challenge – Broader definitions of. . . n n Compilation, OS, verification, certification, etc. Opportunity – Can create custom multi-processor architectures just by downloading bits n After all, a processor is just another circuit Xilinx Virtex II Pro can hold dozens of "soft core" 32 -bit processors, in addition to the four "hard core" Power. PCs Frank Vahid, UC Riverside 20

FPGAs and Cyber-Physical Systems n Tough to predict future of computing n "Victorian planners in 1830 predicted that by 1930 London street traffic would be bogged down under 25 feet of horse manure. " n n n Are FPGAs to microprocessors what cars were to horses? Probably not, but perhaps they are what tails are to horses? We might do well to keep FPGAs in mind as we consider software issues Frank Vahid, UC Riverside 21

e. Blocks: The Wood-and-Nails of the Electronic Sensor World Frank Vahid* Department of Computer Science and Engineering University of California, Riverside vahid@cs. ucr. edu http: //www. cs. ucr. edu/~vahid * Also with the Center for Embedded Computer Systems at UC Irvine This work is being supported by the National Science Foundation Title suggested by a colleague: "e. Blocks – Empowering the People" Frank Vahid, UC Riverside

Seen this Problem? Available n Not! Technology everywhere – Why no good solution? Frank Vahid, UC Riverside 23

Shrinking Processor Size/Cost Enables New Solution Courtesy of Joe Kahn n http: //www. templehealth. org Make sensors smarter n n By adding processor+battery Becomes a "block" easily connected to other blocks Frank Vahid, UC Riverside 24

Shrinking Processor Size/Cost Enables New Solution – e. Blocks Existing component view New "e. Block" view yes/no Button Light Sensor yes/no Magnetic Contact Switch yes/no LED yes/no Beeper yes/no Electric Relay Frank Vahid, UC Riverside 25

e. Blocks n Just connect blocks, and they work n No programming knowledge, no electronics knowledge Frank Vahid, UC Riverside Button yes/no Light Sensor yes/no Beeper LED 26

What's Hard (The Research Part) n (1) Finding right set of building blocks Too many – Overwhelming (too much choice) Too few – Overwhelming (too much configuration) Splitter Toggle 2 -Input Logic Splitter Tripper 3 -Input Logic 123456789 Splitter When A is 2 Prolong (short) When A is yes no AND B is OR Combine Frank Vahid, UC Riverside AND B is OR then the output is yes 4 5: Splitter Prolong (long) 6: . . . 1 2 3 4 5 6 7 8 9 yes no then the output is yes no 3 123456789 4 -Input Logic 123456789 1 2 Yes detector 2 No detector Super. Block 27

What's Hard (The Research Part) (2) Making the blocks understandable People NOT likely to read directions n Combine n Those that do are unlikely to understand Performed extensive user testing (over 500 students, kids, and adults) over two years Example: Combine block B A yes no: A is yes, B is yes A is yes, B is no A is no, B is yes A is no, B is no The output should be yes when: n Phrased truth table A B no no yes yes no A B A is yes, B is yes When the A is yes, B is no the output input is A is no, B is yes should be A is no, B is no out Combine Output no no Logic Block yes yes configurable DIP switch Phrased truth table embedded in sentence A When the A A input is A A yes no B B The output B B should be B out Combine Colored truth table embedded in sentence Most success When A is Frank Vahid, UC Riverside yes no AND B is yes no OR then the output is yes Combine Logic Sentence 28

What's Hard (The Research Part) n (3) Batteries must last years, yet performance should appear continuous n Blocks are off 99. 9% of the time (a) time (b) f (d) Reliability Frank Vahid, UC Riverside < f t f f Connect Responsiveness Latency Disconnect Responsiveness 0. 14 0. 05 0 Block latency (seconds) Fresponsiveness 1 0 0. 25 0. 50 1 2 3 4 5 10 30 60 300 600 1800 Fresponsiveness 1 Flatency 730 Mean time between corrupted packets (days) error < f 0 365 1 0 interpreted as < error 1 Freliability 1 f < (c) Developed custom CAD tool to automatically find the best block parameter settings out of the billions of possibilities t Disconnect response (seconds) 0 0. 10 0. 25 0. 50 1 2 3 4 5 10 30 60 300 600 Developed theory to map e. Block events to continuous time Connect response (seconds) 29

e. Blocks and Embedded Microprocessors n Can greatly simplify coding Button yes/no 1/0 Microprocessor Light Sensor Frank Vahid, UC Riverside yes/no 1/0 30

e. Blocks n Prototypes n n n >100 prototypes, size of deck of cards (trend: smaller) 2 -3 years on 2 AA batteries (trend: longer) Can communicate via wire >1. 5 miles, 150 ft wireless Integer blocks too Early apps: hearing/vision impaired, ageing parents, middle-school projects Support for advanced users Frank Vahid, UC Riverside 31

e. Blocks as a Programming Paradigm n Use virtual blocks in graphical simulator to describe desired sensor system behavior n n Intuitive due to spatial emphasis, not temporal emphasis Automatically compile to code on programmable e. Blocks Partitions Frank Vahid, UC Riverside Programmable e. Blocks 32

e. Block Tool Generates Code n Tool generates C code automatically C Code #include <pic. h> #include “sci. h” main(void) { #include “io. h” ORTA = 0 xff; #include “constants. h” CMCON = 0 x 07; Unsigned char data_val = ERROR; TRISA = 0 x 00; . . . TRISB = 0 x 02; main(void) { asm("CLRWDT"); unsigned I, j; TRISB = 0; . . . } n n . . . } Programmable e. Block Download code to block with click of a button n Ordinary users can “write” programs in minutes Spatial vs. temporally-oriented language n 20 high school graduates: e. Blocks (spatial) vs. LEGO Mindstorms (temporal), 6 example systems, 40 minutes to build Frank Vahid, UC Riverside Type Average Success Rate Mindstorms 0% e. Blocks 54% 33

e. Blocks n 1998: Simple Problem n n No simple solution n n Off-the-shelf solutions costly, hard to find, and/or not customizable Alarm system cost – overkill Connecting existing sensors, logic, transmit/receive, LEDs is hard n n Garage door open at night Electronics, programming 2 -week project: 70% EE/CS unable Countless similar applications go unrealized Why can't I just connect those components like "Lego. TM" blocks? Frank Vahid, UC Riverside transmit AND receive light sensor LED contact switch 34

e. Blocks Example n n "Garage Open at Night" detector <10 minutes to build Light Sensor Detect night-time – use Light Sensor block Magnetic Contact Switch When A is yes no AND B is OR yes no then the output is yes LED Need to indicate garage open at night – use LED block Combine Use Combine block to combine light sensor and contact switch into one Detect garage door open – use Contact Switch block Plug pieces together and the system is done! Frank Vahid, UC Riverside 35

Graphical Simulator w w User specifies and tests block design Java-based simulator n n n User chooses between pallets Blocks added by dragging User is able to configure various blocks by clicking on switches Connections created by drawing lines between blocks User can create, experiment, test and configure design Available e. Blocks Sensors Output Compute/Communications Light Sensor When A is Button Beeper Button yes no AND B is OR yes no then the output is yes When A is yes no AND B is OR yes no then the output is yes Combine Motion Sensor. Combine Green/Red Light Beeper rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 36

Graphical Simulator w w User specifies and tests block design Java-based simulator n User chooses between pallets Available e. Blocks Sensors Output Compute/Communications When A is Button Beeper yes no AND B is OR yes no then the output is yes Motion Sensor. Combine Green/Red Light rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 37

Graphical Simulator w w User specifies and tests block design Java-based simulator n n User chooses between pallets Blocks added by dragging Available e. Blocks Sensors Output Compute/Communications Button When A is Beeper yes no AND B is OR yes no then the output is yes Motion Sensor. Combine Green/Red Light rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 38

Graphical Simulator w w User specifies and tests block design Java-based simulator n n User chooses between pallets Blocks added by dragging Available e. Blocks Sensors Output Compute/Communications Button When A is Button Beeper yes no AND B is OR yes no then the output is yes Motion Sensor. Combine Green/Red Light rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 39

Graphical Simulator w w User specifies and tests block design Java-based simulator n n User chooses between pallets Blocks added by dragging Available e. Blocks Light Sensor Button Sensors Output Compute/Communications When A is Button Beeper yes no AND B is OR yes no then the output is yes Motion Sensor. Combine Green/Red Light rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 40

Graphical Simulator w w User specifies and tests block design Java-based simulator n n User chooses between pallets Blocks added by dragging Available e. Blocks Sensors Output Compute/Communications Light Sensor When A is Button Beeper Button yes no AND B is OR yes no then the output is yes When A is yes no AND B is OR yes no then the output is yes Combine Motion Sensor. Combine Green/Red Light rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 41

Graphical Simulator w w User specifies and tests block design Java-based simulator n n n User chooses between pallets Blocks added by dragging User is able to configure various blocks by clicking on switches Available e. Blocks Sensors Output Compute/Communications Light Sensor When A is Button Beeper Button yes no AND B is OR yes no then the output is yes When A is yes no AND B is OR yes no then the output is yes Combine Motion Sensor. Combine Green/Red Light Beeper rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 42

Graphical Simulator w w User specifies and tests block design Java-based simulator n n User chooses between pallets Blocks added by dragging User is able to configure various blocks by clicking on switches Connections created by drawing lines between blocks Available e. Blocks Sensors Output Compute/Communications Light Sensor When A is Button Beeper Button yes no AND B is OR yes no then the output is yes When A is yes no AND B is OR yes no then the output is yes Combine Motion Sensor. Combine Green/Red Light Beeper rs t in Light Sensor Once Yes, Stays Yes Toggle Yes/No 12345678 9 seconds Prolonger Hide this panel Advanced Mode Welcome to the e. Blocks Simulator! In this area, you’ll find helpful hints on creating your own designs. Click and drag an e. Block off of the “Available e. Blocks” panel to add it to your design. To connect two blocks, click and drag from an output port (colored circle) to an input port (gray circle). A connection can be destroyed by clicking on a connected port. To move a block around the workspace, click and drag its orange area. Blocks can be moved into the trash can to delete them. Green circles indicate that the port is sending a yes, red circles indicate that the port is sending a no, yellow Circles indicate that the port is sending an error signal, and gray circles denote an input port. Frank Vahid, UC Riverside 43