Architecting Testbenches Architecting Testbenches n n n Reusable

Architecting Testbenches

Architecting Testbenches n n n Reusable Verification Components Verilog Implementation VHDL Implementation Autonomous Generation and Monitoring Input and Output Paths Verifying Configurable Designs

Reusable Verification Components n Want to maximize amount of verification code reused across testbenches n n This minimizes the development effort Testbench is made up of 2 components n n Reusable test harness Testcase specific code

Procedural Interfaces for reusable components n For the components to be reusable, must define procedural interfaces that are independent of the detailed implementation n n This way as the interfaces change, the testcases do not. Provide flexibility through thin layers. n n n This gives the ability to have fine control or greater abstraction (byte access vs. . packet access) Do not try to implement all functionality at one level Preserve the procedural interfaces

Development Process for reusable components n Introduce flexibility as needed. n n n This way you don’t spend time on unnecessary functions Use the verification plan to determine what is functions are necessary When you need more complex functions, build upon already verified functions.

VHDL Implementation n Want to refine the monolithic view into a client/server BFM, access and utility functions, and testcases. n n Goal is to obtain a flexible implementation strategy promoting reusability of verification components To change a testcase, just change the control block.

VHDL Implementation – Packaging Bus Functional Procedures n BFM’s can be located in a package to be re-used. n n Driven/monitored signals are passed as signal-class arguments BFM procedure’s are cumbersome to use. n n All signals involved in the transaction must be passed to the procedure Duplication across testbenches, each testbench: n n n Declares all interface signals Instantiates the DUV Properly connects everything

VHDL Implementation – Creating a test harness n Instead – add some abstraction, use a test harness. n n In contains declarations and functionality common to all testbenches Reduces amount of duplicated information n Determine common elements n n n Declaration of the component Declaration of the interface signals Instantiation of the DUV Mapping of Interface signals to the ports on the design Mapping of interface signals to the signal class arguments of the bus-functional procedures

VHDL Implementation – Creating a test harness – (cont) n n Use an intermediate level of hierarchy for test harness encapsulation Have the test harness own the procedures n n All the signals associated with the DUV are local. Testcase control process instructs the local process n Use control signals to communicate to/from the testcase to the test harness

VHDL Implementation – Creating a test harness – (cont) n Control signals: n n n Server signals – signals used in the local (test harness) process Client signals – signals used in the testcase These signals have to be visible to both the client and server (these are located in different architectures) n n Pass as ports in the test harness Global signals in a package

VHDL Implementation – Creating a test harness – (cont) n n Global signals eliminate need for each testbench architecture to declare them Use ‘transaction to synchronize operations n n Won’t miss two consecutive operations (compared to using ‘event) Control signals utilize records n n Minimize maintenance if signals change Minimizes impact on client/server processes

VHDL Implementation – Abstracting the Client/Server Protocol n Client must properly operate the control signals n n n Must wait for the server process to finish the transaction Use transactions, not event based waits Defining the communication protocol on signals between the client/server don’t seem to accomplish much. n n Now instead of having to deal with a physical interface, you have an arbitrary interface Encapsulate this for ease of maintenance n n Removes the client having to know the details of the protocol Change the protocol without having to change testcase

VHDL Implementation – Abstracting the Client/Server Protocol (cont) n n Encapsulate by placing server procedures into a package Client processes are now free from knowing the details of the protocol between the client and server n n To perform an operation, use the appropriate procedure Pass the control signals in the procedure n n Use the package to get access to the global signals and procedures Simply need 2 signals (records) n n One used in communications to the server process One used in communications from the server process

VHDL Implementation – Abstracting the Client/Server Protocol (cont) n What has been gained – signals are still passed from the bus-functional access procedures n n Testcase need not declare all the interface signals to the DUV Testcase need not instantiate and connect the design n Reduction of duplication Regardless of number of signals involved in physical interface, only 2 need to be passed Testcase is completely removed from the physical interface of the DUV n Pins can be removed or added, polarities changed, etc – no effect on existing testcases

VHDL Implementation – Managing Control Signals n Use two control signals (simplest method) n n n Use one control signal – requires additional development effort n n Must have a resolution function for this method Include a mechanism for differentiating value driven from client and server Proper interlocking for parallel procedure calls May have dozens of server processes n n One to send control and synchronization information to the server One to send control and synchronization information to the client Each has its own packages/procedures Use qualified names to prevent identifier collision

VHDL Implementation – Multiple Server Instances n Many designs interface to multiple instances n n If using one BFM instantiated multiple times, need a way to differentiate which instance to use to perform an operation Use an array of control signals n n One pair for each server Can use a generate statement to instantiate the BFM’s

VHDL Implementation – Utility Packages n Utility routines can provide additional level of abstraction n n Composed of procedures Encapsulated in separate packages using lower-level access packages n Example n Use the read/write access functions to perform packet read/write access functions

Autonomous Generation and Monitoring n You now have a properly packaged BFM n These can become active entities n n n Removes tedious task of generating background/random data Removes task of performing detailed response checking Can include variety of tasks n n n Safety checks Data generation Data collection Etc Constant attention from testcase not required n Instead, testcase just configures BFM

Autonomous Stimulus n Protocols may require more data than is relevant for the testcase n n A procedure (send_cell) would inject the cell, but what about the other cycles. n n ATM protocol – when not active cell, must send “dummy” cell Don’t want the testcase to worry about it Add Autonomous stimulus to BFM n Now send_cell would inject the cell at the appropriate time (after current dummy cell is sent) and then return back to the testcase

Autonomous Stimulus (Cont) n For Autonomous stimulus BFM’s, 2 type of procedures: n n Blocking – procedure does not return control back to testcase until BFM is done with function Non-blocking – procedure returns control back to testcase immediately n n For this, actions must be queued. Used to control constraints on the fly

Random Stimulus n Generated data can be random n BFM can contain algorithms to generate random data, in a random sequence, at random intervals n n May want controls (procedures) to BFM to change random constraints Given enough information, the Autonomous BFM can assist in verifying the output of the DUV n Example of packet router – header and filler info could be randomly generated, but expect is a function of this. BFM has knowledge, testcase doesn’t. Use BFM to generate the expect. n Procedures used to start/stop the BFM and fill in the routing table.

Injecting Errors n How do you ensure that your monitors are working? n Could also have BFM to be configured to be enable to generate error conditions to ensure they are detected. n n Example – parity generation Example – CRC generation

Autonomous Monitoring n We have differentiated between monitor and generator based on who is in control of timing (who initiates) n Monitors must be continuously listening n n If not, outputs can be missed If procedures are used to activate monitoring, potential for missed activities n Must have 0 -delays for back to back calls.

Autonomous Monitoring (Cont) n Role of testbench is to detect as many errors as possible! n n n Even errors from within the testbench (Usage errors) If unexpected output operation is not caught, then possible for bad design to go unnoticed Solution – Autonomous Monitors n n Continuously calling monitors is cumbersome – Use autonomous one Continuously monitors outputs – all the time n n n Use queues if needed Testcase can still use procedure, BFM just dequeues from its queues Can have error generated if there are still items in queue at end of test

Autonomous Error Detection n Go one step further and place all checking in Autonomous Monitor n n n Used with environments where data contains description of expected response Could also have stimulus BFM give the monitor the expected response Procedures could pass configuration options

Input and Output Paths n n Each testcase must provide different stimulus and expect different responses Create these differences by configuring test harness in various ways, and providing different data sequences n To this point we have assumed that data was hard coded in the testcase

Programmable Testbenches n n n Most cases, stimulus data and expected response are specified in verification plan and hard coded in testcase Use BFM, in test harness, to apply/receive data using procedures from testcase. Alternative is to have programmable environment n n n Have stimulus/expect read from a file Do not have to recompile for new testcases Common method where you have external ‘C’ program generating data and expect (reference model)

Programmable Testbenches (Cont) n VHDL n n Verilog n n Can read any text file, but very primitive Only read binary/hex values May want to “massage” data before placing into these files

Configurable Files n Packaged BFM’s offer the possibility of making testbench difficult to understand or manage n To minimize possibility, use self-contained BFM’s, controlled by procedures. n n n Only exception may be seed for randomization Functionality of a testcase originates from top-level testcase control description. Non-default configuration settings originate from that point. Avoid external configuration files n n Creates more complicated management task Task grows exponentially with # of files

Configurable Files (Cont) n If you must use files, then: n Do NOT hardcode filenames inside BFM or procedure, instead place the name as a parameter to procedure n n n Not obvious where data is coming from Used in procedure, then top-level testcase shows what file Talk more about output file management this later!

Concurrent Simulation n Other problem with hard coding filenames, concurrent simulation n n Causes collisions between multiple simulations Try to make filenames unique

Compile Time Configuration n n Avoid using Compile time configurations for BFM. Different configuration requires compiling the source code with a different header file n n Makes managing a testcase more complicated – extra files to maintain May make it impossible to run concurrent compiled simulations n n Verilog always compiles before simulation, may make it tempting to use. Minimize # of files. Use a single compile-time configuration

Verifying Configurable Designs n n If different configurations must be verified, architect you environment for this Two type of configurable designs n Soft – Can be changed by a testcase. n n n Randomization constraints This type is implicitly covered by the verification process Hard – Cannot be changed once simulation starts. n n Clock frequency Testbench must be properly designed to support this in a reproducible fashion

Configurable Testbenches n Configure testbench to match DUV n n If a design can be made configurable, so can a testbench Use generics/parameters to achieve this n n Using this approach, almost anything can be configured n n This allows configuration details to be propagated down the hierarchy from the top-level Can be used in signal definitions, generate statements, etc If a component is configurable via generics or parameters, make this part of its interface

Top Level Generics and Parameters n Top-level modules and entities can have generics and parameters n n n Each configurable component has these May want control from top level Top level does not have pins or ports, but it can have generics/parameters n n Can control configurations based on these values Some simulators allow setting these from command line n n Not portable among simulators Can wrap the top level with another layer of hierarchy Verilog can use “defparam” statement VHDL can use configurations to do this

Randomization

Random Environments n “Random” used to describe many environments n n Some teams call testcase generators “random” – they have randomness in the generation process, these are one type Two types: n n n Pre-test randomization During Test Randomization Both utilize autonomous stimulus and autonomous checking

Pre-Test Randomization n n This is a technique used to randomize the controls to the BFM’s Introduce another layer of hierarchy to the verification environment: n n Common testcase Example: DUV has 4 major functions, could do pre-test randomness to randomly pick which of the 4 major functions. n Input to common testcase could be how many to test.

During-Test Randomization n This utilizing the methods of autonomous stimulus and monitoring. For a given interface, done entirely from within a BFM Controlled via procedures from the testcase and/or common testcase.

Random Drivers/Checkers n The most robust random environments use autonomous stimulus and autonomous monitoring n Autonomous stimulus: n n Allows more flexibility and more control Allows the capability to stress the DUV Autonomous monitoring will flag interim errors. Testcase can be stopped as soon as a random input sequence causes an error. Overall quality is determined by verification engineer. n n Scenarios missing from verification plan Checks not implemented correctly or missing

Random Drivers/Checkers n Costs of optimal random environment n n n Code intensive Need experienced engineer to oversee effort to ensure quality Benefits of optimal random environment n n More stress on the logic than any other environment, can include real hardware Find nearly all the bugs n n Most devious Easy ones

Random Drivers/Checkers n Too much randomness can prevent from uncovering problems. n n Want controlled and constrained randomness May want “un-randomizing” controls to be built in n n Hangs due to looping Low activity scenarios Directed tests “Micro-modes” may be put in to the drivers n n Allows user to specify specific scenarios Allows the narrowing of scope for testing

Randomization Seeds n n n Testcase seed is used to seed the pseudonumber generator. The pseudo-number generator is then used for all decision making in the BFM’s. This is chosen at the beginning of the test. n n Usually passed in as a parameter or generic Caution: Watch for seed synchronization across drivers, especially ones instantiated multiple times.