Full Cycle Functional Coverage Coverage Success Stories Avi

Full Cycle Functional Coverage: Coverage Success Stories Avi Ziv Verification Technologies Department IBM Haifa Research Laboratory IBM Verification Seminar 2002

What Is Functional Coverage? • Functional coverage is used to check that important aspects of functionality are tested – Design and implementation specific – Comes in many flavors • Black box and white box • Snapshot and temporal • Local and global scopes • A coverage model is a family of coverage tasks with common denominator and some parameters – In cross-product coverage models the coverage tasks are the cross -product of the values of the parameters (attributes) • Functional coverage is a succinct way to express test plans – Easy to maintain – Automatically measured – Comprehensive IBM Verification Seminar 2002

Motivation • The use of functional coverage in hardware verification is gaining momentum – Available in several coverage tools and verification environments – Much effort is being directed at increasing the efficiency of coverage measurement – More users consider coverage important part of the verification plan • Yet, there are not many functional coverage success stories – Many still do not use it or abuse it – Serious users are disappointed with results • Too much effort • Too little results • A major reason is that using functional coverage the "right" way is more an art than a science IBM Verification Seminar 2002

Coverage Stories • We present here two coverage success stories – Users were very satisfied with coverage process and its contribution to the verification effort • The coverage domains: – Execution units of Power. PC processor – Branch unit of S/390 processor • We describe the coverage process for each domain – Highlight key issues • This is not a scientific study IBM Verification Seminar 2002

First Story: Power. PC Processor Execution Units • Multithreaded • In-order execution • Up to four instructions dispatched per cycle Dispatch Data Fetch B 1 R 1 M 1 S 1 Execute B 2 R 2 M 2 S 2 Write Back B 3 R 3 M 3 Branch (B) Simple Arith (R) Complex Arith (M) S 3 S 4 f S 4 b Load/Store (S) IBM Verification Seminar 2002

Second Story: S/390 Branch Unit • Unit handles branch prediction and execution of branch instructions • Contains – Nine stages complex pipe • More than one instruction at the same time in some stages • instructions can enter the pipe at two places – Branch history tables – and more • 2 PY spent on verification • Done by experts with experience with similar designs • About 100, 000 tests per day IBM Verification Seminar 2002

Coverage Model Definition – Power. PC Pipe • The model was designed to check that all interdependencies between two instructions in the pipeline occur • Model details – Model contains 7 attributes • Type, pipe and stage of first instruction (I 1 , P 1 , S 1) • Same attributes for second instruction (I 2, P 2, S 2) • Type of dependency between the instructions – RR, RW, WR, WW, None – Many restrictions exist • I 1 is simple fixed point Þ P 1 is R or M • P 1 is not S Þ S 1 is 1, 2, or 3 – After restrictions, 4418 tasks are legal IBM Verification Seminar 2002

Coverage Models for Branch Unit • Several models defined – Access to branch tables – Flow of a branch in the pipe – State of the pipe • State of the pipe model – Attributes contain • Location and type of each branch in the pipe in a given cycle • Reset signal – Model size: • Without restrictions ~ 15, 000 • With restrictions ~ 1400 IBM Verification Seminar 2002

Coverage Measurement • Make sure that you measure what you really want and what really happens • Use simpler environment and models to test and debug the measurement system – Hierarchy of models • All instructions • All pipe stages – Controlled simulation IBM Verification Seminar 2002

Analysis of Interdependency Model • After 25, 000 tests 2810 / 4418 tasks were covered (64%) IBM Verification Seminar 2002

Analysis of Interdependency Model • Hole analysis detected two major areas that are lightly covered – Stages S 4 f and S 4 b that are specific to thread switching are almost always empty • Reason: not enough thread switches during tests – The address-base register in the store-and-update instruction is not shared with other registers in the test • Reason: bug in the test generator that didn’t consider the register as a modified register IBM Verification Seminar 2002

Coverage Progress Causes of holes are fixed IBM Verification Seminar 2002

Branch Unit Coverage Progress IBM Verification Seminar 2002

Results • S/390 Branch unit – Better understanding of the tested design • Even before coverage measurement started – Found several tricky performance bugs • Power. PC execution units – – No bugs in the design found Several bugs in the test generator Enhanced test specification Models re-used in follow-on designs • Overall – happy customers!!! IBM Verification Seminar 2002