Basic Hardware Verification Flow Media IC and System

Basic Hardware Verification Flow Media IC and System Lab VLSI Crash Course 2018 Presentor: Yu Sheng Lin

Importance of Verification? ● What it the most important? ● Beautiful and well reviewed code? ● Smart engineer/architecture? ● Complete test s. t. an average engineer cannot ruin the code? 2

Example: ARM Think About Verification ● https: //riscv-basics. com/ ● Verificcation and validation of processor designs can consume 75% of total design time. 3

What Forms Verification? ● I borrow the concept of UVM (Universal Verfication Methodology). ● (We usually call this overall platform as testbench. ) Generate Golden Data Your RTL Drive data to input ports https: //verificationacademy. com Monitor output ports and collect data 4

What Forms Verification? ● Universal Verfication Methodology. ● What does universal means? Universal protocol largely enable code reuse! 5

Some Effective Protocols ● Please review previous slides 6

The Non-RTL Parts ● The non-RTL parts can be implemented without Verilog! ● This is usuallyed called co-simulation (co-sim). 7

Brief Conclusions - A Testbench Must ● Instantiate (make a copy of) your module. ● Driver to send data. ● Monitor to receive and collect them. ● Driver and Monitor might follow specific protocols. ● The collected data are compared by Scoreboard. ○ Golden (Mostly text file in Verilog, or programmatically when co-simed. ) ○ Generated by your RTL module and collected by Monitor. 8

No Need for Verify RTL with Verilog ● Verilog provide external C accesses through VPI. ○ https: //en. wikipedia. org/wiki/Verilog_Procedural_Interface ● Based on C, people develops Java, Python. . . versions. ● AFAIK, there are quite a lot Python based frameworks. ○ myhdl: https: //github. com/myhdl ○ cocotb: https: //github. com/potentialventures/cocotb ○ nicotb: https: //github. com/johnlin/nicotb We focus on this today. 9

Why Should I Write in Testbench Python? ● You have learnt that. ● Why should you write testbench with Verilog if you can write in Python? ● Complete libraries: Numpy and DNN frameworks. . . ● 2 nd~3 rd place popular programming language. ● Easy to learn and to write. ● Not very fast compared with C ○ Do you care about performance in RTL simulation? ● Native asynchronize language support. ● It's cooooool. 10

Python Over Verilog on Verfication ● I show you an example by me. ● You can easily do with Python-Verilog simulation frameworks. ○ Simple GUI verfication with 50 lines Qt (7 -seg display, button). ● Used in NTUEE Digital Circuit Lab #1 11

Interface Reuse → Testbench Reuse ● For example, your code: ○ module Dut( Input input clk, input rst, input irdy, output logic iack, input [10: 0] iint, output logic ordy, input oack, output logic [10: 0] oint ); Dut Output Dataflow graph. ● You can test every module with the same API. 12

Review the Image of UVM & protocol module Dut( input irdy, output logic iack, input [10: 0] iint, output logic ordy, input oack, output logic [10: 0] oint ); 13

Testbench in Python ● input irdy, output logic iack, input [10: 0] iint, output logic ordy, input oack, output logic [10: 0] oint Master Slave ● Group the red parts as a Master. ○ Master. Send(100) ● Group the blue parts as a Slave. ○ Slave. Expect(101) ● The same rules apply for Verilog-based verification. ○ But Verilog is less dynamic and ugly. 14

Asynchronize（非同步）Issues ● Will this always work? ○ def main(): Master. Send(100) Slave. Expect(101) ● What if Send depends on Expect? ● In Verilog we have can fork easily. ○ initial while (1) begin irdy = 1; @(posedge clk). . . end initial while (1) begin oack = 0; @(posedge clk). . . end ● In modern languages such as Python and Java. Script, asynchronized execution is common. 15

Asynchronize（非同步）Issues ● The asynchronize code could actually look like. ● def main(): yield clk yield from Master. Send(100) Slave(callback=[Scoreboard. Expect, print]) 1. yield clk = @(posedge clk) in Verilog 2. yield from = the function version wait 3. callback: call the callbacks whenever data is monitored. a. print the monitored data and b. put the data to the checker (Scoreboard). Google Python "yield" and "yield from" for more information. 16

How to Combine Python and Verilog ● Verilog controls the scheduling of Python code ● The magic of yield keyword. def Master(): while True: . . . yield def Slave(): while True: . . . yield Fork([Slave, Master]) VPI while (1) begin @(posedge clk) $Python. Run() end wire oack, iint. . . 17

How to Combine Python and Verilog ● Python code can read or write the Verilog signals. ○ Do you remember the +access+rw for ncverilog. def Master(): while True: . . . yield def Slave(): while True: . . . yield Fork([Slave, Master]) VPI while (1) begin @(posedge clk) $Python. Run() end wire oack, iint. . . 18

A Small Co-Sim Example with Nicotb ● You can find example here https: //github. com/johnlin/nicotb ● module Dut( input clk, input rst, input irdy, output logic iack, input [10: 0] iint, output logic ordy, input oack, output logic [10: 0] oint ); ● If you send an N, then you can expect 0, 1, 2, 3…, N-1 19

Prepare a Verilog Wrapper `Pos(rst_out, rst) `Pos. If(ck_ev, clk, rst) always #1 clk = ~clk; initial begin clk = 0; rst = 1; #1 $Nicotb. Init(); #11 rst = 0; #10 rst = 1; #1000 $display("Timeout"); $Nicotb. Final(); $finish; end Declare your module here! While you might not understand it, this template requires almost no modifications! Declare your module here! assign oack = ordy && ocanack; Dut dut(clk, rst, irdy, iack, iint, ordy, oack, oint); 20

Prepare a Verilog Wrapper `Pos(rst_out, rst) `Pos. If(ck_ev, clk, rst) always #1 clk = ~clk; initial begin clk = 0; rst = 1; #1 $Nicotb. Init(); #11 rst = 0; #10 rst = 1; #1000 $display("Timeout"); $Nicotb. Final(); $finish; end You don't need to understand all of them and mostly you only modify the module to test and simulation timeout. assign oack = ordy && ocanack; Dut dut(clk, rst, irdy, iack, iint, ordy, oack, oint); 21

(Prepare a Makefile) Modify according to your path (irun = ncverilog) NICOTB=~/nicotb/lib IRUN=/opt/CAD/INCISIV/cur/tools. lnx 86/bin/64 bit/irun %: %. sv GLOG_logtostderr=1 TEST=$(if $(TEST), $@) TOPMODULE=$(if $(TOPMODULE), $@) PYTHONPATH=$(NICOTB)/python: `pwd` $(IRUN) +access+rw -loadvpi $(NICOTB)/cpp/nicotb. so: Vpi. Boot $(NICOTB)/verilog/Utils. sv $< You must prepare XXX_test. py and XXX_test. sv under current directory (see previous pages). Also, the top level testbench module is XXX_test. Then, just type make XXX. The lab in these days is an example. 22

File Preparation Progress ● Prepare the RTL for design under test (DUT). ○ O Not covered today. ● Script for loading Python automatically (Makefile). ● System. Verilog wrapper. ○ Copy from examples. ● Python testbench. ○ O O X The most important one! 23

Bridging Python and Verilog (Events) ● Add these lines in Python ○ rst_out_ev, ck_ev = Create. Events(["rst_out", "ck_ev", ]) ● Add this lines in Verilog ○ `Pos(rst_out, rst) `Pos. If(ck_ev, clk, rst) ● This means, whenever a clk posedge in Python, ck_ev is triggered ○ Mostly your submodules use 1 reset and clock and you just copy it. ● That is, ○ Python yield ck_ev = Verilog @posedge clk ● However, simply writing yield in Python doesn't make thing easier. ○ We will explain later. 24

Bridging Python and Verilog (Wires) ● API to connect Verilog wires ○ my_data_bus = Create. Bus(( ("", "a_signal", (4, 2)), ("dut", "sig"), ("dut", "sig 2"), )) hierarchy: toplevel → "" name: a_signal shape: (4, 2) ● The Verilog to be connected ○ logic [7: 0] a_signal [4][2]; DUT my_dut(. clk(clk), . sig(sig) ) 25

Bridging Python and Verilog (Wires) ● API to connect Verilog wires ○ my_data_bus = Create. Bus(( ("", "a_signal", (4, 2)), ("dut", "sig"), ("dut", "sig 2"), )) ● The Verilog to be connected ○ logic [7: 0] a_signal [4][2]; DUT my_dut(. clk(clk), . sig(sig) ) hierarchy: "dut" (nested: "dut. a. b") name: sig shape: not a array 26

Bridging Python and Verilog (Wires) ● API to connect Verilog wires ○ my_data_bus = Create. Bus(( ("", "a_signal", (4, 2)), You don't have to connect sig 2 to ("dut", "sig"), the toplevel since Verilog allows ("dut", "sig 2"), hierarchy access. )) ● The Verilog to be connected ○ logic [7: 0] a_signal [4][2]; DUT my_dut(. clk(clk), . sig(sig) ) 27

Access Verilog Wires in Python ● my_data_bus = Create. Bus(( ("", "a_signal", (4, 2)), ("dut", "sig"), ("dut", "sig 2"), )) my_data_bus. Read() my_data_bus. Write() ● my_data_bus. a_signal. value → Numpy array of shape (4, 2) ● my_data_bus. sig. value → Numpy array of shape (1, ) ● my_data_bus. sig 2. value → Numpy array of shape (1, ) 28

Some Notes ● Make use of Python unpack easily. ○ a = Create. Bus(A) b = Create. Bus(B) c = Create. Bus(C) a, b, c = Create. Buses([A, B, C]) ● This is the method of Nicotb, every framework has its method connecting Verilog to Python (or whatever). 29

Short Conclusions ● Connect to Verilog wires. ○ a = Create. Bus(( ("", "a_signal", (4, 2)), ("dut", "sig"), ("dut", "sig 2"), )) a, b, c = Create. Buses([. . . , . . . ]) a. Read() a. Write() ● Access Verilog wires (32 -bit limitation). ○ a. a_signal. value → 4 x 2 Numpy array ○ a. values. a_signal → Equivalent 30

Short Conclusions ● Connect to Verilog events ○ rst_out_ev, ck_ev = Create. Events(["rst_out", "ck_ev", ]) yield ck_ev ○ `Pos(rst_out, rst) `Pos. If(ck_ev, clk, rst) @(posedge clk) 31

Is That All? ● If you ONLY know these, it doesn't make things easier. ○ yield clk a. data. value[3: 4] = 100 a. Write() yield clk ● Protocol based verification is the true keypoint! 32

Review Our Initial Problem ● Our toplevel module. ○ module Dut( Input input clk, input rst, input irdy, output logic iack, input [10: 0] iint, output logic ordy, input oack, output logic [10: 0] oint ); Dut Output Dataflow graph. ● Input data: N ● Output data: 0, 1, 2, . . . , N ● Test data: input 0, 8, 7 33

Convert Bus into Protocol ● First, you need to create the buses ○ irdy iack iint ordy oack oint = = = Create. Bus((("dut", "irdy"))) "iack"))) "iint"))) "ordy"))) "oack"))) "oint"))) ● Then, construct the classes in Python. ○ master = Two. Wire. Master(irdy, iack, iint, ck_ev, A=1, B=5) slave = Two. Wire. Slave( ordy, oack, oint, ck_ev, callbacks=[test. Get] ) 34

Check at Scoreboard ● Generate golden data for Scoreboard (not today's main issue). ○ Ns = np. array([0, 8, 7], dtype=np. int 32) # this create a column vector [0, 0, 1, 2, . . . , 8, 0, 1, . . . , 7] golden = np. concatenate([ np. arange(N+1, dtype=np. int 32) for N in Ns ])[: , np. newaxis] test. Expect((golden, )) ● What should I Expect? 35

Data Format in Nicotb ● Look at the data bus. ○ ordy = Create. Bus((("dut", "ordy"))) oack = Create. Bus((("dut", "oack"))) oint = Create. Bus((("dut", "oint"))) ● Slave put that to Scoreboard. ○ The actual code is slightly more complex than above pages, while you can just copy and modify. st = Stacker(1+9+8, [bg. Get]) # PS: 1+9+8+=18 bg = Bus. Getter(callbacks=[st. Get]) slave = Two. Wire. Slave( ordy, oack, oint, ck_ev, callbacks=[bg. Get] ) ● oint is a bus with one scalar (aka, not an array), so you get a tuple of size ([18, 1], ). 36

Data Format in Nicotb ● Look at the data bus. ○ oint = Create. Bus(( ("dut", "aaa") ("dut", "bbb", (1, )) ("dut", "ccc", (12, 3)) )) ● In this example, the tuple size is ([18, 1], [18, 12, 3]). 37

Back to Our Example ● Look at the data bus. ○ golden = np. concatenate([ np. arange(N+1, dtype=np. int 32) for N in Ns ])[: , np. newaxis] test. Expect((golden, )) ● golden is a tuple of size ([18, 1], ). ● np. concatenate generate a array of size [18, ], namely a row vector. ● The [: , np. newaxis] is the standard method converting Numpy row vector to a column vector (vertical one). 38

We are Almost Done! ● Prepare the RTL for design under test (DUT). O ● Script for loading Python automatically (Makefile). ● System. Verilog wrapper. ● Python testbench. O △ ○ Prepare input data and golden (it's your task). ○ Send the data. ○ Check the data at scoreboard. X O O O X O 39

Drive Verilog Wire In Python (Quite Easy!) Probability = A/B (default = 1/5) master = Two. Wire. Master(irdy, iack, iint, ck_ev, A=1, B=5) values = master. values Access data bus by name def it(): for N in Ns: values. iint[0] = N yield values. iint is a Numpy array of size (1, ) yield from master. Send. Iter(it()) This randomly drive the input. yield from master. Send. Iter(it(), latency=100) This drive data every 100 cycles. 40

Note values = master. values def it(): This part is only Python generator syntax, for N in Ns: it has no relationship with waiting Verilog values. iint[0] = N posedge!!! yield values yield from master. Send. Iter(it()) yield from master. Send. Iter(it(), latency=100) Every 100 cycles (ignore probability) 41

Conclusions ● Introduce the idea behind System. Verilog UVM. ● With Python, you can do the same thing much easily. ● We introduce Nicotb today. ○ Document: https: //johnlin. github. io/nicotb/ ● And there are many choices. ○ myhdl: https: //github. com/myhdl ○ cocotb: https: //github. com/potentialventures/cocotb 42

Practice Time Media IC and System Lab VLSI Crash Course 2018 Presentor: Shih Yi Wu

Lab 2 ISE (Top) Img N_IMG =16 8 b pixel_data [3] Img 80 pixel_tag Counter img_tag, img_type img_num, img_sum 120 2 b o_type Sorter 5 b o_tag 44

Sorter_testbench(already done (ref)) clk rst img_valid img_tag Host (Testbench) img_type Sorter img_num img_sum o_valid 2 b o_type 5 b o_tag 45

Counter_testbench(todo) clk rst pixel_valid pixel_ready Host (Testbench) pixel_data [3] pixel_tag Counter img_valid img_tag img_type img_num img_sum 46

Top_testbench(todo) + top module wire connection(todo) clk ISE (Top) rst Counter pixel_valid pixel_ready Host (Testbench) 8 b pixel_data [3] ? 5 b pixel_tag o_valid Counter 2 b o_type 5 b o_tag 47

RTL verification (Review) ● Prepare the RTL for design under test (DUT). ● Script for loading Python automatically (Makefile). ● System. Verilog wrapper. ● Python testbench. ○ Prepare input data and golden. ○ Send data. ○ Check the data at scoreboard. 48

Python testbench -- sending data (1/3) 1. Connecting Verilog Wire (Ref: Bridging python and Verilog) function Create. Bus, Create. Buses python: xxx_test. py my_data_bus = Create. Bus(( ("", "a_signal", (4, 2)), ("dut", "sig"), ("dut", "sig 2"), )) system verilog: xxx_test. sv logic [7: 0] a_signal [4][2]; DUT my_dut(. clk(clk) ) system verilog: DUT. sv module DUT( input clk, input sig 2 ). . . 49

Python testbench -- sending data (2/3) 2. Determine the protocol 2 - wire protocol ( 1 - wire protocol ( i_valid, i_rdy, i_data ) = Create. Buses([( i_valid, i_data ) = Create. Buses([( (("dut", "src_valid"), ), (("dut", "src_ready"), ), (("dut", "i_data"), ), ]) ]) Two. Wire. Master(i_valid, i_rdy, i_data, ck_ev) One. Wire. Master(i_valid, i_data, ck_ev) 50

Python testbench -- sending data (2/3) 2. Determine the protocol Two. Wire. Master(i_valid, i_ready, i_data, ck_ev, A=1, B=5) One. Wire. Master(i_valid, i_data, ck_ev, A=1, B=5) Two. Wire. Slave(o_valid, o_ready, o_data, ck_ev, callbacks=[bg. Get]) One. Wire. Slave(o_valid, o_data, ck_ev, callbacks=[bg. Get]) 51

Python testbench -- sending data (2/2) 3. Sending data from python to Verilog (Ref: Drive Verilog Wire In Python) ( i_valid, i_data ) = Create. Buses([( (("dut", "src_valid"), ), (("data"), ), ]) master = One. Wire. Master(i_valid, i_data, ck_ev) mdata = master. values. . . mdata[0] = Img_r system verilog: xxx_test. sv logic [7: 0] data [3]; mdata[1] = Img_g mdata[2] = Img_b … 52

Python testbench -- Check the data at scoreboard Ref: Check at Scoreboard Notice the oder of the wire and the golden numebr scb = Scoreboard("ISE") (N_IMG == 16) test = scb. Get. Test(f"Sorter") st = Stacker(N_IMG, callbacks=[test. Get]) bg = Bus. Getter(callbacks=[st. Get]) # Construct slave (monitor) and connect slave to the scoreboard. slave = One. Wire. Slave(ovalid, odata, ck_ev, callbacks=[bg. Get]) # Check the data at slave. golden # This create a tuple of two column vectors of size 16. # The first one is o_tag, and the second one is o_type. test. Expect((answer[: , 1, np. newaxis], answer[: , 0, np. newaxis])) 53

Things you should know before you start 1. 2. 3. 4. There is a reference python testbench (. /sim/Sorter_test. py) type make sorter under. /sim → it should work! All golden data are prepared (. /sim/My. Model. py) Files you need to edit and the execution cmd: Verilog: ===========================. /sim/Counter_test. py (Testbench) → make Counter. /design/Top. v (Wire Connection). /sim/Top_test. py (Testbench) → make USE_VERILOG=true top System Verilog: ===========================. /sim/Counter_test. py (Testbench) → make Counter. /design/Top. sv (Wire Connection). /sim/Top_test. py (Testbench) → make top 54