Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex
Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications Melanie Berg MEI Technologies- NASA/GSFC Radiation Effects and Analysis Group
Agenda: Asynchronous FIFO (AFIFO) Basics: Operation and Architecture Single Event Upsets (SEUs) and Xilinx FPGA Error Signatures Mitigating Radiation Effects is AFIFOs with Triple Modular Redundancy (TMR): Xilinx Generated Intellectual Property (IP) Cores User Customized Design Conclusion Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 2 Melanie Berg MEI Technologies/ NASA Goddard REAG
AFIFO Basics Used to pass data between two separate clock domains Basic AFIFO Write Domain Read Domain WCLK Each domain has its own: Clock Enable Data port Address Pointer + Protection Domains share memory space Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 RCLK WEN REN DIN DOUT FULL EMPTY Write Read Protection Flag Page 3 Melanie Berg MEI Technologies/ NASA Goddard REAG
AFIFO Operation: Internal Address Pointers Write Pointer (WP) 0000000 There are 2 internal address pointers: Write Pointer (WP) Read Pointer (RP) Pointers increment independently Pointers are inaccessible by user Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Read Pointer (RP) 0000000 At Start: RP = WP= 0 Empty = ‘ 1’ Full = ‘ 0’ If Empty RP will not increment If Full WP will not increment Page 4 Melanie Berg MEI Technologies/ NASA Goddard REAG
What’s the Problem Regarding Critical Mission Application? Asynchronous nature of the AFIFO leads to non-deterministic behavior Empty and full flag can be generated on either cycle(t) or cycle(t+1) Redundant AFIFOs can be out of step with each other Critical Missions will generally require redundancy + mitigation due to SEEs IP Cores have limited protection capability if redundancy is required Designers must be aware of asynchronous corner cases when inserting redundancy and mitigation “Design for Radiation Effects, ” M. Berg Short Course MAPLD 2008; “A Comparative Study of Field Programmable Gate Array Error Cross Sections: Putting Data into Perspective, ” M. Berg, C. Perez April, SEE 2008 Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 5 Melanie Berg MEI Technologies/ NASA Goddard REAG
AFIFO Operation: Internal FULL Detection Example (WP) (RP) Traditional: nnnnn yyyyy Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Look Ahead: Page 6 Melanie Berg MEI Technologies/ NASA Goddard REAG
WCLK Example: Comparing WP and RP Across Asynchronous Clock Domains WP changes @ WCLKedge+(tclk→q) RP changes @ RCLKedge+(tclk→q) Compare: RP Crosses into WCLK Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Metastable Event Page 7 Melanie Berg MEI Technologies/ NASA Goddard REAG
Reducing the Number of Metastable bits in Compare Path Problem: With binary encoding too Binary Gray many address bits can change 000 simultaneously during an increment: 000 001 11111→ 00000 Resolution: Use gray encoding Only used to cross domains Only one bit changes per clock cycle Comparison must still be performed in binary Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 010 011 100 101 110 111 010 111 100 Page 8 Melanie Berg MEI Technologies/ NASA Goddard REAG
Scheme: Reducing the Number of Metastable bits in Compare Path using Converters Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 9 Melanie Berg MEI Technologies/ NASA Goddard REAG
Xilinx AFIFO IP Core Block Diagram sly u o i v Pre ted a r t s Illu le p m a Ex Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 10 Melanie Berg MEI Technologies/ NASA Goddard REAG
Single Event Upsets (SEUs) and Xilinx FPGA Error Signatures Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 After Nikkei Science, Inc. Page 11 Melanie Berg MEI Technologies/ NASA of Japan, by. Goddard K. Endo REAG
Xilinx Architecture Basics Configuration programmable switches static during operation Logic: Switches during operation Performs expected functionality Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 12 Melanie Berg MEI Technologies/ NASA Goddard REAG
Programmable Switch Implementation and (SEU) Susceptibility SRAM (Reprogrammable Configuration) Sensitive Volume Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 13 Melanie Berg MEI Technologies/ NASA Goddard REAG
Xilinx XQR 4 VSX 55: Radiation Test Data Xilinx Consortium: VIRTEX-4 VQ STATIC SEU CHARACTERIZATION SUMMARY: April/2008 Probability Error Rate LEO GEO Configuration Memory: XQR 4 VSX 55 Pconfiguration 7. 43 4. 2 Combined SEFIs per device PSEFI 7. 5 x 10 -5 2. 7 x 10 -5 For non-mitigated designs the most significant upset M Berg, Trading ASIC and FPGA Considerations for System factor is: Insertion; IEEE Nuclear Science Radiation Effects Conference 2009 Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 14 Melanie Berg MEI Technologies/ NASA Goddard REAG
Configuration Upset Effects I 1 I 2 I 3 I 4 D Q I 1 I 2 I 3 I 4 LUT Functional Fault R O U T I N G M A T R I X I 1 I 2 I 3 I 4 LUT Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 LUT Page 15 Melanie Berg MEI Technologies/ NASA Goddard REAG
DFF Configuration Upset Effects I 1 I 2 I 3 I 4 LUT R O U T I N G M A T R I X Clock Turns off I 1 I 2 I 3 I 4 Functional Fault and Potential Metastability LUT Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 LUT Page 16 Melanie Berg MEI Technologies/ NASA Goddard REAG
Routing Configuration Upset Effects I 1 I 2 I 3 I 4 LUT R O U T I N G M A T R I X I 1 I 2 I 3 I 4 LUT Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 LUT Page 17 Melanie Berg MEI Technologies/ NASA Goddard REAG
AFIFO Error Signatures Will Read Invalid locations Can write over unread data Fixing the configuration bit will not fix the address pointer Most importantly, a large amount of data can be lost nnnnnnnnnnnnn xxxxxxx Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 18 Melanie Berg MEI Technologies/ NASA Goddard REAG
AFIFO IP Core Error Signatures Address Pointers are internal to FIFO and cannot be fixed Pointer Corruption (and data loss) is not detectable Error and Full Flags become invalid WCLK WEN nnnn DIN FULL Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 RCLK REN xxxx DOUT EMPTY Page 19 Melanie Berg MEI Technologies/ NASA Goddard REAG
Mitigating Radiation Effects in AFIFOs with Triple Modular Redundancy (TMR) Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 20 Melanie Berg MEI Technologies/ NASA Goddard REAG
IP Cores: Manual Core TMR WCLK Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 RCLK WEN REN DIN DOUT FULL EMPTY Page 21 Melanie Berg MEI Technologies/ NASA Goddard REAG
Implications of Core Triplication: Lack of Visibility and Control Inability to correct read and internal address pointers More than one core will go bad over time Scrubbing will not fix logic state of IP Core (need a voter) Inactivity will accumulate and the system will eventually not be valid Asynchronous issues negate effectiveness of mitigation Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 V O T I N G FPGA IP CORE M A T R I X FPGA IP CORE Page 22 Melanie Berg MEI Technologies/ NASA Goddard REAG
User Customized Design: Creating Correction Control WCLK Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 RCLK WEN REN DIN DOUT FULL EMPTY Page 23 Melanie Berg MEI Technologies/ NASA Goddard REAG
User AFIFO Custom Design and Global TMR (GTMR) Triplicate everything in circuit: Clocks, I/O, logic, resets… Place Voter after every DFF (or only DFFs with feedback) Level of triplication can vary (e. g. no clock redundancy, I/O redundancy, or only voters in feedback paths) Mitigated User Non. Mitigated Design Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 24 Melanie Berg MEI Technologies/ NASA Goddard REAG
Mitigation Windows Significantly Decrease Susceptibility Small mitigation windows = Greater Protection= Low Probability of upset DFF 0 1 2 DFF V DFF 0 1 2 DFF 3 4 5 DFF V V DFF 3 4 5 DFF V Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 6 7 8 DFF V Page 25 Melanie Berg MEI Technologies/ NASA Goddard REAG
Upper-Bound Error Prediction: Xilinx FPGA XTMR What about PConfiguration ? ? ? After GTMR, SEUs become insignificant MBUs may be insignificant (still under investigation) Assumes Unmitigated SEFIs are the most predominant source: Error Rates migrate from Days to Years Source: M. Berg: “Trading ASIC and FPGA Considerations for System Insertion”, NSREC Short Course 2009, Quebec Canada Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 26 Melanie Berg MEI Technologies/ NASA Goddard REAG
Global TMR (GTMR)… Continued Redundancy and Mitigation created via a tool: Tool creates redundancy and inserts voters postsynthesis (at gate level) Can be difficult to validate tool TMR insertion did anything break? is everything mitigated as expected? Commercially Available tools: Xilnx … XTMR Mentor Graphics (soon to be available) Will have a formal checker to validate RTL matches tool Redundancy and Mitigation created via RTL: RTL implementation Best for design reviews Difficult to implement and may be impractical Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 27 Melanie Berg MEI Technologies/ NASA Goddard REAG
User Custom AFIFO GTMR Design and Address Correction Address Pointers are protected Voters=Correction 1 Voter per Address Bit Voters Feed Memory Block Memory Triplication is Optional Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 28 Melanie Berg MEI Technologies/ NASA Goddard REAG
GTMR Example: Non Redundant Flag Generation Logic Path… We Start Here! USER non. Redundant Design Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 29 Melanie Berg MEI Technologies/ NASA Goddard REAG
Full Flag Example: Non-voted Asynchronous Domain Crossings No Feedback… No Voter Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 30 Melanie Berg MEI Technologies/ NASA Goddard REAG
Asynchronous Triggering… Lagging or Leading Circuitry Downstream Voter Insertion Too Late! State Machine FLAG Trigger LAG Behind other two FLAG Trigger Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 State Machine V O T I N G M A T R I X Will Never Match Page 31 Melanie Berg MEI Technologies/ NASA Goddard REAG
Asynchronous Flags and Voter Placement Voter Placed downstream on Voted Flag can feedback-only Trigger logic at DFFs… flag is the same clock not mitigated cycle State Machine starts early Stateof Machines Domains are. Allout Sync Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 start Page 32 Melanie Berg MEI Technologies/ NASA Goddard REAG
GTMR User Implementation Solution #1: Insert Voters after Metastability Filter Voters Will Correct out of sync Redundant String Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 33 Melanie Berg MEI Technologies/ NASA Goddard REAG
GTMR User Implementation Solution #2: Insert Voters after Flag DFFs Voters can be inserted by hand or…. Voters can be instantiated by making the flag DFF have feedback (use of enable) Voters Will Correct Out of sync Redundant String Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 34 Melanie Berg MEI Technologies/ NASA Goddard REAG
Risk: XTMR Clock Skew and Race Conditions Example: Red Clock domain (as seen by DFFs) has skew relative to other domains Race condition on feedback path if its delay is faster than clock skew Tfb Total Skew: Tsk = Sio + Sroute + Sint_max Sio : Skew Measured at Input Boundary Sroute: Skew of route from Input to Clock tree buffer Sint_max: Static Timing Analysis max Skew Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 35 Melanie Berg MEI Technologies/ NASA Goddard REAG
Advantages of Custom Designed AFIFOs with GTMR Addresses can be corrected with voters Addresses will be synchronized for all three clock domains FIFOs will always write to the same location Availability of operation is prolonged due to internal correction capability Design reviews are enhanced because actual AFIFO implementation can be evaluated in the RTL (VHDL or Verilog). Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 36 Melanie Berg MEI Technologies/ NASA Goddard REAG
Conclusion AFIFOs pass data from one clock domain to another AFIFO’s SEU susceptibility: Configuration can have several upsets per day (Xilinx) Large amounts of data can be lost AFIFO IP Cores have no internal correction and can be significantly susceptible to SEUs Implementing a customized GTMR AFIFO design reduces SEU susceptibility and availability of operation Asynchronous concerns must be addressed: Most tools will not handle these situations Design reviewers must validate domain crossings Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 37 Melanie Berg MEI Technologies/ NASA Goddard REAG
Thank You… NASA Electronics Parts and Packaging (NEPP) Kenneth La. Bel and NASA REAG Chris Perez Mark Friendlich Embedding Asynchronous FIFO Memory Blocks in Xilinx Virtex Series FPGAs Targeted for Critical Space System Applications MAPLD 2009 Page 38 Melanie Berg MEI Technologies/ NASA Goddard REAG
- Slides: 38