Industry Pulse Trends in Functional Verification Harry Foster

  • Slides: 76
Download presentation
Industry Pulse: Trends in Functional Verification Harry Foster Chief Scientist Verification Design Verification Technology

Industry Pulse: Trends in Functional Verification Harry Foster Chief Scientist Verification Design Verification Technology Memo. CODE 2013

Extrapolating From Current Conditions Disregards Future Innovation “In 1910, in the early history telephony,

Extrapolating From Current Conditions Disregards Future Innovation “In 1910, in the early history telephony, a Bell telephone statistician projected a massive ramp-up in switchboard operator jobs as telephone use grew, until “every woman in America” would be required. ” Source: Future Savvy: Identifying trends to Make Better Decisions, Manage Uncertainty, and Profit From Change Adam Gordon, 2008 © 2013 Mentor Graphics Corp. 2 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Functional Verification Market According to EDAC 38% Growth Between 2010 2012 965 Millions ($)

Functional Verification Market According to EDAC 38% Growth Between 2010 2012 965 Millions ($) 1000 800 700 600 400 200 0 2012 EDAC: Market Statistics Service 2007 Annual Summary Report © 2013 Mentor Graphics Corp. 3 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Functional Verification Market According to EDAC Millions ($) Simulation 13% Growth Emulation 94% Growth

Functional Verification Market According to EDAC Millions ($) Simulation 13% Growth Emulation 94% Growth Formal 31% Growth 430 400 380 365 300 200 170 190 130 100 0 2012 EDAC: Market Statistics Service 2007 Annual Summary Report © 2013 Mentor Graphics Corp. 4 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

2012 Wilson Research Group Functional Verification Study n Conducted by Wilson Research Group —

2012 Wilson Research Group Functional Verification Study n Conducted by Wilson Research Group — Commissioned by Mentor Graphics — Format followed 2002, 2004 Collett studies for trend analysis, as well as the 2007 Far. West Research Study n Worldwide study — Overall confidence of 95% plus/minus 4. 05% n This was a blind study! — To eliminate any bias in the results n This was a balanced study! Wilson Research Group — No single vendor dominated responses © 2013 Mentor Graphics Corp. 5 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Who Participated In The Survey Participant’s market segment 60% Non-FPGA 2012 50% Study Participants

Who Participated In The Survey Participant’s market segment 60% Non-FPGA 2012 50% Study Participants FPGA 2012 40% 30% 20% 10% 0% ASIC Vendor Fabless IC Vendor IC Manufacturer Systems Company Design Services Company IP Vendor Company Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 6 HF, Memo. CODE, 2013 www. mentor. com Company Confidential Other

Who Participated In The Survey Participant’s job title 60% Non-FPGA 2012 Study Participants 50%

Who Participated In The Survey Participant’s job title 60% Non-FPGA 2012 Study Participants 50% FPGA 2012 40% 30% 20% 10% PP SU D CA © 2013 Mentor Graphics Corp. 7 HF, Memo. CODE, 2013 www. mentor. com Company Confidential R HE NE GI EN T OR AR W FT SO Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission OT ER ER AN EN E M GI NE AG ER ER AM TE AR EM ST SY LE IT CH GI EN ON TI IF IC A VE R AD EC ER NE ER GN SI DE E AR W RD HA T 0%

Overview n Beyond Theory n Beyond Standards n Beyond the Status Quo © 2013

Overview n Beyond Theory n Beyond Standards n Beyond the Status Quo © 2013 Mentor Graphics Corp. 8 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Beyond Theory in Terms of Rising Complexity BEYOND THEORY © 2013 Mentor Graphics Corp.

Beyond Theory in Terms of Rising Complexity BEYOND THEORY © 2013 Mentor Graphics Corp. HF, 9 Memo. CODE, 2013 www. mentor. com Company Confidential

Difference Between Theory and Practice In theory there is no difference between theory and

Difference Between Theory and Practice In theory there is no difference between theory and practice, but in practice there is. © 2013 Mentor Graphics Corp. 10 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Difference Between Theory and Practice Theory: Everything is clear, but nothing works. © 2013

Difference Between Theory and Practice Theory: Everything is clear, but nothing works. © 2013 Mentor Graphics Corp. 11 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Difference Between Theory and Practice: Everything works, but nothing is clear. © 2013 Mentor

Difference Between Theory and Practice: Everything works, but nothing is clear. © 2013 Mentor Graphics Corp. 12 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Difference Between Theory and Practice The problem is sometimes theory meets practice: Nothing works

Difference Between Theory and Practice The problem is sometimes theory meets practice: Nothing works and nothing is clear. © 2013 Mentor Graphics Corp. 13 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Beyond Theory in Terms of Rising Complexity n What does this really mean? n

Beyond Theory in Terms of Rising Complexity n What does this really mean? n What makes things complex? n How do we measure complexity? © 2013 Mentor Graphics Corp. 14 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

What Makes Something Complex? n System consisting of many interconnected parts — Examining the

What Makes Something Complex? n System consisting of many interconnected parts — Examining the individual parts tells you nothing about the system n Complex does not necessarily mean complicated © 2013 Mentor Graphics Corp. 15 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Designs are Getting More Complex 40% 2007: Mean 90 nm 2010: Mean 65 nm

Designs are Getting More Complex 40% 2007: Mean 90 nm 2010: Mean 65 nm 2012: Mean 45 nm Study Participants 35% 30% 2007 2010 2012 25% 20% 15% 10% 5% or sm al le r 0. 04 5µ 9µ 0. 0 3µ 0. 1 5µ 0. 1 8µ 0. 1 0. 06 5µ 0. 03 2µ 0. 25 µ or LA R G ER 0% Process Geometry Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 16 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Designs are Getting More Complex Number of gates of logic and datapath, excluding memories

Designs are Getting More Complex Number of gates of logic and datapath, excluding memories 30% About a 1/3 rd of designs below 5 M gates About a 1/3 rd of design between 5 M - 20 M gates About a 1/3 rd of designs great than 20 M gates Non-FPGA Study Participants 25% 2002 15% 2007 2010 10% 2012 5% 60 M > 60 M 40 M > > 20 M -4 0 M 20 M 10 M > > 5 M - 1 M > - 10 M 5 M 1 M 50 0 K > 10 0 K or LE SS 50 0 K 0% Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 17 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Designs are Getting More Complex Mean number of gates of logic and datapath, excluding

Designs are Getting More Complex Mean number of gates of logic and datapath, excluding memories trends Non-FPGA Mean Design Size Gates (M) 12 11. 1 10 8 6. 1 6 4 2. 7 2 0. 4 0 2002 2007 2010 2012 Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 18 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Designs are Getting More Complex 79% of designs contain one or more embedded processors

Designs are Getting More Complex 79% of designs contain one or more embedded processors 60% Non-FPGA Study Participants 50% 2004 40% 2007 2010 2012 30% 28% 22% 21% 10% 13% 10% 6% 0% NONE 1 2 3 4 5 or MORE Number of Embedded Processors for Non-FPGA Designs Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 19 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Designs are Getting More Complex Non-FPGA Mean number of Embedded Processors 3 Mean number

Designs are Getting More Complex Non-FPGA Mean number of Embedded Processors 3 Mean number of embedded processors continues to rise 2. 25 1. 96 2 1. 46 1. 06 1 0 2004 2007 2010 2012 Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 20 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

FPGAs are Getting Complex Too! 56% of FPGAs contain one or more embedded processors

FPGAs are Getting Complex Too! 56% of FPGAs contain one or more embedded processors 44% Survey Participants 40% 38% FPGA 30% Non-FPGA 28% 20% 22% 21% 10% 13% 10% 11% 6% 4% 1% 2% 0% NONE 1 2 3 4 5 or MORE Number of embedded processors Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 21 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

How do we measure complexity? n Computational complexity theory used in computer science Operations

How do we measure complexity? n Computational complexity theory used in computer science Operations O(1) O(logn) O(nlogn) O(n^2) O(2^n) O(n!) Elements n There are no generally accepted metrics! © 2013 Mentor Graphics Corp. 22 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Is bug density a good proxy? Channel Encoder TX Decoder Compressed Audio n Single,

Is bug density a good proxy? Channel Encoder TX Decoder Compressed Audio n Single, sequential data streams — — — Floating point unit Graphics shading unit DSP convolution unit MPEG decode. . . Sequential data streams 1 x number of bugs HF, Memo. CODE, 2013 PHY RX n Multiple, concurrent data streams — — — Cross bar Bus traffic controller DMA controller Standard I/F (e. g. , PCIe). . . Concurrent data streams 5 x number of bugs -Ted Scardamalia, internal IBM study 23 Data Link Layer © 2013 Mentor Graphics Corp. www. mentor. com Company Confidential

Concurrency is Complicated to Verify Packet-Based Design From Fabric Tx Transaction Layer Packet Reformater

Concurrency is Complicated to Verify Packet-Based Design From Fabric Tx Transaction Layer Packet Reformater Retry Buffer To PHY Arbiter Data Link Layer Packet Reformater Rx From Rx Channel © 2013 Mentor Graphics Corp. 24 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Expensive Prohibitive Critical Threshold Affordable Cost/Effort Maybe effort is a good proxy? Complexity ©

Expensive Prohibitive Critical Threshold Affordable Cost/Effort Maybe effort is a good proxy? Complexity © 2013 Mentor Graphics Corp. 25 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Verification Consumes Majority of Project Time Non-FPGA Study Participants Total Project Time Spent in

Verification Consumes Majority of Project Time Non-FPGA Study Participants Total Project Time Spent in Verification 25% 2007: Mean 49% 2010: Mean 56% 2012: Mean 56% 2007 2010 2012 15% 10% 5% 0% 1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% Time (Percent) Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 26 HF, Memo. CODE, 2013 www. mentor. com Company Confidential >80%

More and More Verification Engineers Mean peak number of design vs. verification engineers ~

More and More Verification Engineers Mean peak number of design vs. verification engineers ~ 1 -to-1 ratio of peak design and verification engineers 11% 58% 8. 4 Verification Engineers 7. 8 8. 1 8. 5 Design Engineers 2007 2010 2012 7. 6 4. 8 4% 5% Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 27 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Where Verification Engineers Spend Their Time More time spent in debug than any other

Where Verification Engineers Spend Their Time More time spent in debug than any other task! 36% 4% Test Planning Testbench Development 16% 23% 22% Creating and Running Test Debug Other Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 28 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Designers Doing More and More Verification Design Engineer Project Time 2007 - 2012 54%

Designers Doing More and More Verification Design Engineer Project Time 2007 - 2012 54% 15 % In cre as e 53% 47% 46% 2007 2012 Doing Design 2007 2012 Doing Verification Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 29 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Time Designers Spends in Design vs. Verification At this rate… In 25 years, ALL

Time Designers Spends in Design vs. Verification At this rate… In 25 years, ALL of a designer’s time will be devoted to verification 100 Time (Percent) 80 60 Time Design Engineers Spends Doing: 40 Design 20 Verification 0 2007 2012 2017 2022 2027 2032 2037 Project Time 2007 - 2037 © 2013 Mentor Graphics Corp. 30 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Design Reuse Trends 41% 2007 Non-FPGA Study Participants 40% 33% 30% 35% 2012 28%

Design Reuse Trends 41% 2007 Non-FPGA Study Participants 40% 33% 30% 35% 2012 28% 22% 20% 13% 15% 10% 0% NEW LOGIC REUSED LOGIC (Developed in-house) PURCHASED IP ANALOG, RF AND/OR MIXED SIGNAL Design Composition Source: Wilson Research Group and Mentor Graphics. © 2013 Mentor Graphics Corp. 31 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Verification Reuse Mean testbench composition trends Non-FPGA Study Participants 50% 40% 50% 39% 41%

Verification Reuse Mean testbench composition trends Non-FPGA Study Participants 50% 40% 50% 39% 41% 2007 44% 2012 30% 20% 17% 8% 10% 0% NEW REUSED FROM OTHER DESIGNS Testbench Composition ACQUIRED EXTERNALLY Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 32 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

With All This Effort, How are We Doing? © 2013 Mentor Graphics Corp. 33

With All This Effort, How are We Doing? © 2013 Mentor Graphics Corp. 33 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Project’s Schedule Completion Trends 35% 2007: 67% behind schedule 2010: 66% behind schedule 2012:

Project’s Schedule Completion Trends 35% 2007: 67% behind schedule 2010: 66% behind schedule 2012: 67% behind schedule Study Participants 30% 25% 2007 20% 2010 2012 15% 10% 5% 0% More than 10% EARLY ON-SCHEDULE Ahead of schedule 20% 40% >50% BEHIND SCHEDULE Behind Schedule Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 34 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

FPGA vs. Non-FPGA Completion Trends 35% 30% Non-FPGA: 67% behind schedule Study Participants 25%

FPGA vs. Non-FPGA Completion Trends 35% 30% Non-FPGA: 67% behind schedule Study Participants 25% 20% Non-FPGA 15% 10% 5% 0% More than 10% ON-SCHEDULE 20% 40% >50% BEHIND EARLY SCHEDULE Non-FPGA vs. FPGA completion compared to project's original schedule Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 35 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Required Number of Spins Non-FPGA Study Participants 50% 40% 2004 2007 30% 2010 2012

Required Number of Spins Non-FPGA Study Participants 50% 40% 2004 2007 30% 2010 2012 20% 10% 0% 1 FIRST SILICON SUCCESS 2 3 4 5 6 Number of Required Spins 7 SPINS or MORE Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 36 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Types of Flaws Non-FPGA Study Participants 60% 50% 2004 40% 2007 2010 30% 2012

Types of Flaws Non-FPGA Study Participants 60% 50% 2004 40% 2007 2010 30% 2012 20% 10% TO ER O TH PS O R D TH IR O W M R FI PA – PA TI M IN G – G IN M TI FA ST E AR W O TH TO O SL LI R O R LD YI E IG -S ED IX M EL IA BI R TE IN AL N CO ER W TY CE FA IO N SU M PT TA SS O CR PO G IN N TU N LK IT CU CI R AL O AN O IC G LO G CL O R FU N CT I O CK I N N AL G 0% Trends in Types of Flaws Resulting in Respins * Multiple answers possible Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 37 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Root Cause of Functional Flaws Non-FPGA Study Participants 90% 80% 2002 70% 2004 2007

Root Cause of Functional Flaws Non-FPGA Study Participants 90% 80% 2002 70% 2004 2007 60% 2010 50% 2012 40% 30% 20% 10% 0% DESIGN ERROR CHANGES IN INCORRECT or FLAW IN SPECIFICATION INCOMPLETE INTERNAL SPECIFICATION REUSED BLOCK, CELL, MEGACELL or IP FLAW IN EXTERNAL IP BLOCK or TESTBENCH OTHER Root Cause of Functional Flaws * Multiple answers possible Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 38 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Cost of Find Functional Flaws Silicon Debug, Doug Josephson and Bob Gottlieb, (Paul Ryan)

Cost of Find Functional Flaws Silicon Debug, Doug Josephson and Bob Gottlieb, (Paul Ryan) D. Gizopoulos (ed. ), Advances in Electronic Testing: Challenges and Methodologies, Springer, 2006 © 2013 2010 Mentor Graphics Corp. Company Confidential 39 HF, Memo. CODE, 2013 www. mentor. com

Beyond arguing over who won the standards war BEYOND STANDARDS © 2013 Mentor Graphics

Beyond arguing over who won the standards war BEYOND STANDARDS © 2013 Mentor Graphics Corp. HF, 40 Memo. CODE, 2013 www. mentor. com Company Confidential

Standardization of Languages Non-FPGA Study Participants 80% System. Verilog grew 8. 3% between 2010

Standardization of Languages Non-FPGA Study Participants 80% System. Verilog grew 8. 3% between 2010 and 2012 2007 60% 2010 2012 40% 20% 0% VHDL Verilog Synopsys Vera System C System. Verilog Specman e C/C++ Languages Used for Verification (testbenches) Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 41 HF, Memo. CODE, 2013 www. mentor. com Company Confidential OTHER Testbench

System. Verilog Adoption by Design Size Non-FPGA Study Participants 100% 89% 80% 60% 71%

System. Verilog Adoption by Design Size Non-FPGA Study Participants 100% 89% 80% 60% 71% 59% 40% 20% 0% < 5 M 5 - 20 M System. Verilog Adoption by Design Size (Gate Count Excluding Memories) > 20 M Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 42 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Standardization in Base Class Libraries 486%UVM growth between 2010 and 2012 46%UVM projected growth

Standardization in Base Class Libraries 486%UVM growth between 2010 and 2012 46%UVM projected growth in the next twelve months Non-FPGA Study Participants 40% 2010 2012 30% 20% 10% 0% Accellera UVM OVM Mentor AVM Synopsys Cadence VMM RVM e. RM Cadence URM Testbench Methodologies and Base-Class Libraries Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 43 HF, Memo. CODE, 2013 www. mentor. com Company Confidential Other

Standardization of the So. C Verification Process n Ten years ago, IC/ASIC verification was

Standardization of the So. C Verification Process n Ten years ago, IC/ASIC verification was partitioned into two main steps: Block-Level Verification Full Chip Integration Verification © 2013 Mentor Graphics Corp. 44 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Standardization of the So. C Verification Process n Emerging from ad hoc to systematic

Standardization of the So. C Verification Process n Emerging from ad hoc to systematic processes IP Block-Level Verification Subsystem Interconnect Verification So. C System Integration Verification © 2013 Mentor Graphics Corp. 45 HF, Memo. CODE, 2013 www. mentor. com Application / SW Verification Company Confidential

Beyond surviving by maintaining the status quo BEYOND THE STATUS QUO © 2013 Mentor

Beyond surviving by maintaining the status quo BEYOND THE STATUS QUO © 2013 Mentor Graphics Corp. HF, 46 Memo. CODE, 2013 www. mentor. com Company Confidential

The Verification Paradox n A good verification process lets you get the most out

The Verification Paradox n A good verification process lets you get the most out of best-in-class verification tools Start Tools Ad Hoc Processes 6 -9% Cost Increase Start Process Tools 20 -30% Cost Savings Source: Cisco Momentum Research Group © 2013 Mentor Graphics Corp. 47 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Standardization of the So. C Verification Process Block-Level Verification Interconnect Verification Integration Verification ©

Standardization of the So. C Verification Process Block-Level Verification Interconnect Verification Integration Verification © 2013 Mentor Graphics Corp. 48 HF, Memo. CODE, 2013 www. mentor. com Application / SW Verification Company Confidential

Use of Advanced Verification Techniques 2007 2012 48% Code coverage 70% 37% Assertions 68%

Use of Advanced Verification Techniques 2007 2012 48% Code coverage 70% 37% Assertions 68% 40% Functional coverage 71% 41% Constrained-Random Simulation 62% 0% 20% 40% 60% Non-FPGA Study Participants Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 49 HF, Memo. CODE, 2013 www. mentor. com Company Confidential 80%

Directed vs Constrained-Random Simulation 100% Non-FPGA Study Participants 80% 49% 56% 60% Directed Constrined-Random

Directed vs Constrained-Random Simulation 100% Non-FPGA Study Participants 80% 49% 56% 60% Directed Constrined-Random 40% 20% 44% 16% se ea r c In 51% 0% 2010 2012 Mean Directed vs. Constrained-Random Simulation Trends Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 50 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Larger Designs Use More Formal 41% Non-FPGA Study Participants 40% 30% 26% 20% 10%

Larger Designs Use More Formal 41% Non-FPGA Study Participants 40% 30% 26% 20% 10% 0% < 5 M 5 - 20 M > 20 M Formal Property Checking Adoption by Design Size (Gate Count Excluding Memories) Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 51 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

The Evolution of Formal Technology High Effort Formal Property Checking Automated Applications Fully Automatic

The Evolution of Formal Technology High Effort Formal Property Checking Automated Applications Fully Automatic Formal Low Effort 1990 s 2000 s Today Formal Experts Everyone © 2013 Mentor Graphics Corp. 52 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Standardization of the So. C Verification Process Block-Level Verification Interconnect Verification A 57 Integration

Standardization of the So. C Verification Process Block-Level Verification Interconnect Verification A 57 Integration Verification A 57 © 2013 Mentor Graphics Corp. 53 HF, Memo. CODE, 2013 Application / SW Verification www. mentor. com Company Confidential

Standardization of the So. C Verification Process Block-Level Verification — — — Interconnect Verification

Standardization of the So. C Verification Process Block-Level Verification — — — Interconnect Verification Integration Verification IP Blocks connectivity Access all memories Access all registers, such as control Configurations work Functional scenarios and use-cases Verify multiple clock domain crossings © 2013 Mentor Graphics Corp. 54 HF, Memo. CODE, 2013 Application / SW Verification www. mentor. com Company Confidential

Mean Number of Clock Domains by Design Size Mean Number of Clock Domains 14

Mean Number of Clock Domains by Design Size Mean Number of Clock Domains 14 11. 48 12 9. 36 10 8 6 5. 21 4 2 0 < 5 M 5 - 20 M > 20 M Mean Number of Clock Domains by Design Size (Gates Excluding Memories) Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 55 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Standardization of the So. C Verification Process Block-Level Verification — — — Interconnect Verification

Standardization of the So. C Verification Process Block-Level Verification — — — Interconnect Verification Integration Verification Boot OS Load System Drivers Run Application SW © 2013 Mentor Graphics Corp. 56 HF, Memo. CODE, 2013 Application / SW Verification www. mentor. com Company Confidential

So. C Design & Verification Involves Lots of SW It’s the software, stupid! -Gary

So. C Design & Verification Involves Lots of SW It’s the software, stupid! -Gary Smith $100 ($ M) $80 $60 $40 $20 $0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Total HW Engineering Costs + EDA Tool Costs Total SW Engineering Costs + ESA Tool Costs Source: ITRS 2010, Impact of Design Technology on So. C Consumer Portable Implementation Cost © 2013 Mentor Graphics Corp. 57 HF, Memo. CODE, 2013 www. mentor. com Company Confidential 2012

As Design Sizes Increase…Emulation Up, FPGA Prototyping Down in 2012 60% 57% 50% Non-FPGA

As Design Sizes Increase…Emulation Up, FPGA Prototyping Down in 2012 60% 57% 50% Non-FPGA Study Participants < 5 M 5 - 20 M > 20 M 50% 45% 40% 32% 30% 28% 20% 18% 10% 0% HW Acceleration/Emulation FPGA Prototyping Adoption by design size for those doing HW acceleration/emulation and FPGA Prototyping Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study © 2013 Mentor Graphics Corp. 58 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Integrated Simulation/Emulation/Software Verification Environments Emerge Design under Testbench Acceleration System Level OVM/UVM System. C/C++

Integrated Simulation/Emulation/Software Verification Environments Emerge Design under Testbench Acceleration System Level OVM/UVM System. C/C++ Virtual Prototype Monitors SW Debug Processor Debug Assertions & Checkers JTAG Protocol Solutions Virtual Devices & Transactors Physical Devices © 2013 Mentor Graphics Corp. 59 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Coverage and Power Across all aspects of verification Block-Level Verification Subsystem Interconnect Verification So.

Coverage and Power Across all aspects of verification Block-Level Verification Subsystem Interconnect Verification So. C Integration Verification © 2013 Mentor Graphics Corp. 60 HF, Memo. CODE, 2013 www. mentor. com System Application / SW Verification Company Confidential

The Rising Importance of Coverage Signoff criteria trends WHEN ALL TESTS DOCUMENTED IN THE

The Rising Importance of Coverage Signoff criteria trends WHEN ALL TESTS DOCUMENTED IN THE VERIFICATION PLAN ARE COMPLETE AND PASS WHEN THE PROJECT PLAN SAYS SIGN-OFF, ASSUMING VERIFICATION LOOKS OK WHEN COVERAGE SAYS WE HAVE ACHIEVED A TARGET WHEN THE EMULATED OR PROTOTYPED DESIGN IS WORKING IN-SITU WHEN THE RATE OF BUGS FOUND PER WEEK DROPS BELOW A SPECIFIED GOAL WHEN WE CAN NO-LONGER THINK OF ANY MORE TESTS TO WRITE 2007 2012 WHEN THE PROJECT PLAN SAYS SIGN-OFF, REGARDLESS OF STATUS OTHER 0% 10% 20% 30% 40% Non-FPGA Study Participants * Multiple answers possible Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 61 HF, Memo. CODE, 2013 50% www. mentor. com Company Confidential

Unified Coverage Interoperability Standard New Accellera UCIS Standard Announced at DAC 2012 Formal Simulation

Unified Coverage Interoperability Standard New Accellera UCIS Standard Announced at DAC 2012 Formal Simulation Emulation UCIS API Coverage Database Analysis Testplan © 2013 Mentor Graphics Corp. 62 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Trends in Power Management Verification 67% of the industry actively manages power Hypervisor/OS control.

Trends in Power Management Verification 67% of the industry actively manages power Hypervisor/OS control. . . Application-level powe. . . Operation in each syste. . . Interactions between. . . Hardware power contr. . . Transitions between. . . Power domain state. . . Power domain power. . . 0% 20% 40% 60% 80% Aspects of Non-FPGA Power Managed Design That Are Ve Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 63 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Power Trends 35% About 10% of power managed designs perform no power-aware simulation! Non-FPGA

Power Trends 35% About 10% of power managed designs perform no power-aware simulation! Non-FPGA Study Participants 30% 25% 20% 15% 10% 5% 0% e n No re % % % % o 9 9 9 9 -2 -3 -4 -1 -5 -6 -7 rm o % 1% % % % 3 4 % 20 10 50 60 70 80 Percentage of total simulations that were power-aware Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. HF - January 2013 Master Set, WRG & MG Study Results www. mentor. com Company Confidential

Power Trends 35% Median Verification Resources: 20% -29% Non-FPGA Study Participants 30% 25% 20%

Power Trends 35% Median Verification Resources: 20% -29% Non-FPGA Study Participants 30% 25% 20% 15% 10% 5% 0% ne No % ore 9% 9% 9% -29% 3 4 5 6 7 1 -9 m or 1% 10% % 3% 4% 0 0 0 % 20% 5 6 7 80 Percentate of Verification Resources Focused on Power Management Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. HF - January 2013 Master Set, WRG & MG Study Results www. mentor. com Company Confidential

Power Trends Notation used to describe power intent 27% 24% 22% 20% 15% 10%

Power Trends Notation used to describe power intent 27% 24% 22% 20% 15% 10% 6% 6% 5% ry ta ie 00 pr -2 ro 01 /P al rn te In U PF 2. 0 U (I PF EE 1. E 0 18 (A CP F 2. cc el le ra 9) ) 0 1 1. CP F 1. 0 0% CP F Non-FPGA Study Participants 25% * Multiple answers possible Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission © 2013 Mentor Graphics Corp. 66 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Beyond the Status Quo Standardization of the So. C Verification Process IP Block-Level Verification

Beyond the Status Quo Standardization of the So. C Verification Process IP Block-Level Verification Subsystem Interconnect Verification So. C System Integration Verification © 2013 Mentor Graphics Corp. 67 HF, Memo. CODE, 2013 www. mentor. com Application / SW Verification Company Confidential

The Productivity Gap BACK TO THE FUTURE © 2013 Mentor Graphics Corp. HF, 68

The Productivity Gap BACK TO THE FUTURE © 2013 Mentor Graphics Corp. HF, 68 Memo. CODE, 2013 www. mentor. com Company Confidential

Design Productivity Gap Capacity Silicon Density Design Productivity Time Source: SEMATECH © 2013 Mentor

Design Productivity Gap Capacity Silicon Density Design Productivity Time Source: SEMATECH © 2013 Mentor Graphics Corp. 69 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Verification Productivity Gap Capacity Silicon Density Design Productivity Verification Productivity Source: SEMATECH Time ©

Verification Productivity Gap Capacity Silicon Density Design Productivity Verification Productivity Source: SEMATECH Time © 2013 Mentor Graphics Corp. 70 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Closing The Verification Gap Reuse Acceleration ? Abstraction Methodology © 2013 Mentor Graphics Corp.

Closing The Verification Gap Reuse Acceleration ? Abstraction Methodology © 2013 Mentor Graphics Corp. 71 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Managing Complexity #include "systemc. h" SC_MODULE(adder) // module (class) declaration { sc_in<int> a, b;

Managing Complexity #include "systemc. h" SC_MODULE(adder) // module (class) declaration { sc_in<int> a, b; // ports sc_out<int> sum; void do_add() // process { sum = a + b; } SC_CTOR(adder) // constructor { SC_METHOD(do_add); // register do_add to kernel sensitive << a << b; // sensitivity list of do_add } }; 100 s of lines of TLM Millions of Lines of RTL 100 s of Millions of Gates Billions of Transistors 72 72 HF, Memo. CODE, 2013 © 2013 Mentor Graphics Corp. www. mentor. com Company Confidential

Productivity Gains Through Abstraction Simulation Function arguments C/C++ Transaction Untimed TLM Transaction Timed TLM

Productivity Gains Through Abstraction Simulation Function arguments C/C++ Transaction Untimed TLM Transaction Timed TLM 10, 000 x (1 min) 1, 000 x 100 x clk Cycle Accurate Protocol clk Protocol 73 73 HF, Memo. CODE, 2013 RTL 10 x 1 x (7 days) © 2013 Mentor Graphics Corp. www. mentor. com Company Confidential

Bug Prevention vs. Bug Hunting What’s the advantage of System. C compared with RTL?

Bug Prevention vs. Bug Hunting What’s the advantage of System. C compared with RTL? There are two different aspects – quality and time schedule. Today, a full chip on System. C will run around 10 MHz, and you will never reach that speed using RTL or a lower-level abstraction. It’s similar to a prototype speedup. Previously with RTL designs, our bug rate was in the range of 10 to 50 bugs per square millimeter. Now we are at less than one bug per millimeter squared. So we have both quality and speed of development. Source: EETimes, 2007, Laurent Ducousso, who manages intellectual-property (IP) verification for STMicroelectronics’ Home Entertainment Division © 2013 Mentor Graphics Corp. 74 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

Summary n Beyond theory in terms of rising complexity n Beyond arguing over who

Summary n Beyond theory in terms of rising complexity n Beyond arguing over who won the standards wars n Beyond surviving by maintaining the status quo © 2013 Mentor Graphics Corp. 75 HF, Memo. CODE, 2013 www. mentor. com Company Confidential

© 2013 Mentor Graphics Corp. www. mentor. com Company Confidential

© 2013 Mentor Graphics Corp. www. mentor. com Company Confidential