Building an Internet Router P 33 Handout 1

Building an Internet Router (P 33) Handout 1: What’s a router? Class project and logistics Dr Andrew W. Moore andrew. moore@cl. cam. ac. uk Thank you supporters: Michaelmas 2009 Building an Internet Router (P 33)Handout 1 1

Some logistics Web page: http: //www. cl. cam. ac. uk/teaching/0910/P 33/ TAs: David Miller (Hardware) david. miller@cl. cam. ac. uk Stephen Kell (Software) stephen. kell@cl. cam. ac. uk Hot-spare TAs: Phil Watts (Hardware) philip. watts @cl. cam. ac. uk Chris Smowton (Software) chris. smowton@cl. cam. ac. uk Grades: Mphil (ACS) – Pass / Fail - based on a mark out of 100 All others (DTC) – Mark out of 100 Michaelmas 2009 Building an Internet Router (P 33)Handout 1 2

Some more logistics Class Mailing list: bir-list@cl. cam. ac. uk think of this as a self-help mailing list for all of you Staff Mailing list: bir-staff@cl. cam. ac. uk Submission mail: bir-tick@cl. cam. ac. uk Group aliases can appear if you require them. 3

What is Routing? R 3 R 1 A R 4 D B E D C R 2 Destination Next Hop D R 3 E R 3 F R 5 F 4

What is Routing? R 3 R 1 A 1 4 Ver 20 bytes B C R 4 32 16 HLen T. Service Fragment ID TTL D Total Packet Length E Flags Fragment Offset D Protocol Header Checksum Source Address R 2 Destination Next Hop Destination Address D Options (if any)R 3 E R 3 Data F R 5 F 5

What is Routing? R 3 A R 1 R 4 D B C E R 2 R 5 F 6

Points of Presence (POPs) POP 2 A POP 1 POP 4 B C POP 3 D E POP 5 POP 6 POP 7 POP 8 F 7

Where High Performance Routers are Used (10 Gb/s) R 1 R 2 R 5 R 4 R 3 R 8 R 9 R 10 R 7 R 11 R 14 R 13 (10 Gb/s) R 6 R 15 (10 Gb/s) R 12 R 16 8 (10 Gb/s)

What a Router Looks Like Cisco CSR-1 Juniper T 320 19” Capacity: 92 Tb/s Power: 4. 7 k. W 6 ft Capacity: 320 Gb/s Power: 2. 9 k. W 3 ft 2. 5 ft 9

Basic Architectural Components of an IP Router Routing Protocols Routing Table Hardware Forwarding Switching Table Software Management & CLI Control Plane Datapath per-packet processing 10

Per-packet processing in an IP Router 1. Accept packet arriving on an incoming link. 2. Lookup packet destination address in the forwarding table, to identify outgoing port(s). 3. Manipulate packet header: e. g. , decrement TTL, update header checksum. 4. Send packet to the outgoing port(s). 5. Buffer packet in the queue. 6. Transmit packet onto outgoing link. 11

Generic Router Architecture Header Processing Data Hdr Lookup Update IP Address Header IP Address ~1 M prefixes Off-chip DRAM Queue Packet Data Hdr Next Hop Address Table Buffer Memory ~1 M packets Off-chip DRAM 12

Generic Router Architecture Data Hdr Header Processing Lookup IP Address Update Header Address Table Data Hdr Data Memory. Hdr Header Processing Lookup IP Address Buffer Manager Buffer Address Table Data Hdr Buffer Memory Address Table Data Hdr Buffer Manager Update Header Buffer Manager Buffer Memory 13

The P 33 Project: Build a 4 -port x 1 GE Router Exception Processing Routing Protocols Routing Table Hardware Forwarding Switching Table Software Management & CLI Software Team Develop s/w in Linux on PC in lab Hardware Team Develop h/w in Verilog on Net. FPGA board in lab 14

1 2 Build basic router 3 4 Command Line Routing Protocol Interface (PWOSPF) 5 Integrate with H/W Interoperability 6 Wow us! Management & CLI Exception Processing Routing Protocols Management & CLI Routing Table Exception Processing Emulated h/w in VNS software hardware Management & CLI Routing Exception Protocols Processing Routing Table Routing Protocols Routing Table Emulated h/w in VNS 21/Oct Emulated h/w in VNS 28/Nov • Innovate and add! • Presentations • Judges Management & CLI 5/Nov Exception Processing Routing Protocols Routing Table 11/Nov 18/Nov 10/Jan Forwarding Switching Table Learning Environment Modular design Testing Forwarding Switching Table 4 -port non-learning 4 -port learning IPv 4 router switch forwarding path 21/Oct 28/Nov 5/Nov Integrate with S/W Interoperability 11/Nov Wow us! 18/Nov 10/Jan 15

What is the Net. FPGA? Networking Software running on a standard PC CPU Memory PCI A hardware accelerator built with Field Programmable Gate Array driving Gigabit network links PC with Net. FPGA 1 GE 1 GE Memory 1 GE Net. FPGA Board

Running the Router Kit User-space development, 4 x 1 GE line-rate forwarding Management (including CLI) CPU Exception Processing Memory PW-OSPF user kernel Routing Table PCI “Mirror” Fwding Table Packet Buffer FPGA IPv 4 Router Memory 1 GE 1 GE

Next steps v Explore the web page (http: //www. cl. cam. ac. uk/teaching/0910/P 33/) Decide if you still want to take the class v Build a team of 3 people: 2 s/w and 1 h/w v Come to SW 02 on Tuesday 10 am… v We will meet with you every week during class time. 18

Using the nf-test machines v Each group is allocated an nf-test machine {nf-test 1, nf-test 2, nf-test 3, . . . }. nf. cl. cam. ac. uk v These machines are headless (no KVM) and located in SW 02 (alcove) Login via ssh; set your own password and go v SW 02 is a busy classroom – please respect others v We suggest running “vncserver” and then using “vncviewer”; as a useful way to leave jobs/desktop running and reconnect if you need to move location. If you need to gain physical access to an nf-test machine Do it quietly! You maybe refused entry in the afternoons – don’t take it personally.

nf-test machine rules Abuse the machines – that’s the end of this module for you – zero tolerance policy. v only connect eth 0 to the cisco (access) switch v these machines run a firewall for a reason v Ø if it doesn’t work for you, lets fix the rules v Play nicely with each other; your interoperability marks depend on it.

nf-test machine information Warning: Files stored on nf-test machines are not backed up - and may be lost at any time. v You must make sure that you regularly copy your files into your REAL (computer laboratory) home directory as a back-up v Machines fail – they usually fail 24 hours before the big deadline! v

Tree Structure NF 2 bin (scripts for running simulations and setting up the environment) bitfiles lib (contains the bitfiles for all projects that have been synthesized) (stable common modules and common parts needed for simulation/synthesis/design) projects (user projects, including reference designs) Building an Internet Router (P 33) Michaelmas 2009 22

Tree Structure (2) lib C (common software and code for reference designs) java (contains software for the graphical user interface) Makefiles (makefiles for simulation and synthesis) Perl 5 (common libraries to interact with reference designs and aid in simulation) python (common libraries to aid in regression tests) scripts (scripts for common functions) verilog (modules and files that can be reused for design) Building an Internet Router (P 33) Michaelmas 2009 23

Tree Structure (3) projects doc (project specific documentation) (contains file to include verilog modules from lib, include and creates project specific register defines files) tests used to test generated regress (regression bitfiles) non-library verilog code used for src (contains synthesis and simulation) sw (all software parts of the project) synth verif (contains user. xco files to generate cores and Makefile to implement the design) (simulation tests) Building an Internet Router (P 33) Michaelmas 2009 24

State Diagram to Verilog (1) MA C Rx. Q CP U Rx. Q MA C Tx. Q CP U Tx. Q Input Arbiter Output Port Lookup Output Queues MA C Tx. Q CP U Tx. Q Building an Internet Router (P 33) Michaelmas 2009 MA C Tx. Q CP U Tx. Q 25

Inter-module Communication data Module i ctrl wr Module i+1 rdy Building an Internet Router (P 33) Michaelmas 2009 26

State Diagram to Verilog (2) • Projects: – Each design represented by a project Format: NF 2/projects/<proj_name> e. g. , NF 2/projects/bir_starter – Consists of: • Verilog source • Simulation tests • Hardware tests • Libraries • Optional software – What is Missing? • refer to the web page “Implement four port hard-wired switch” is the start. Building an Internet Router (P 33) Michaelmas 2009 27

State Diagram to Verilog (3) • Projects (cont): – Pull in modules from NF 2/lib/verilog • Generic modules that are re-used in multiple projects • Specify shared modules in project’s include/lib_modules. txt – Local src modules override shared modules – bir_starter: • Local in_arb_regs. v input_arbiter. v output_port_lookup. v • Everything else: shared modules Building an Internet Router (P 33) Michaelmas 2009 28

Testing: Simulation (1) • Simulation allows testing without requiring lengthy synthesis process • Net. FPGA provides Perl simulation infrastructure to: – Send/receive packets • Physical ports and CPU – Read/write registers – Verify results • Simulations run in Model. Sim/VCS Building an Internet Router (P 33) Michaelmas 2009 29

Testing: Simulation (2) • Simulations located in project/verif • Multiple simulations per project – Test different features • Example: – bir_starter/verif/test_fwd_source. Port 0 • Test forwarding of packets from port 0 to port 1 Building an Internet Router (P 33) Michaelmas 2009 30

Testing: Simulation (3) • Example functions: – nf_PCI_read 32(delay, batch, addr, expect) – nf_PCI_write 32(delay, batch, addr, value) – nf_packet_in(port, length, delay, batch, pkt) – nf_expected_packet(port, length, pkt) – nf_dma_data_in(length, delay, port, pkt) – nf_expected_dma_data(port, length, pkt) – make_IP_pkt(length, da, sa, ttl, dst_ip, src_ip) Building an Internet Router (P 33) Michaelmas 2009 31

Running Simulations • Use command nf 2_run_test. pl – Optional parameters • --major <major_name> • --minor <minor_name> • --gui (starts the default viewing environment) test_fwd_source. Port 0 major minor • Problems? check the environment variables reference your project • NF 2_DESIGN_DIR=/root/NF 2/projects/<project> • PERL 5 LIB=/root/NF 2/projects/<project>/lib/Perl 5: /root/NF 2/lib/Perl 5: Building an Internet Router (P 33) Michaelmas 2009 32

Running Simulations • When running modelsim interactively: – Click "no" when simulator prompts to finish – Changes to code can be recompiled without quitting Model. Sim: • bash# cd /tmp/$(whoami)/verif/<projname>; make model_sim • VSIM 5> restart -f; run -a – Problems? do ensure $NF 2_DESIGN_DIR is correct Building an Internet Router (P 33) Michaelmas 2009 33

Synthesis • To synthesize your project – Run make in the synth directory (/root/NF 2/projects/bir_starter/synth) Building an Internet Router (P 33) Michaelmas 2009 34

Regression Tests • Test hardware module • Perl Infrastructure provided includes: – Read/Write registers – Read/Write tables – Send Packets – Check Counters Building an Internet Router (P 33) Michaelmas 2009 35

Example Regression Tests • For a Router… – Send Packets from CPU – Longest Prefix Matching Misses – Packets dropped when queues overflow – Receiving Packets with IP TTL <= 1 – Receiving Packets with IP options or non IPv 4 – Packet Forwarding – Dropping packets with bad IP Checksum Building an Internet Router (P 33) Michaelmas 2009 36

Perl Libraries • Specify the Interfaces – eth 1, eth 2, nf 2 c 0 … nf 2 c 3 • Start packet capture on Interfaces • Create Packets – MAC header – IP header – PDU • Read/Write Registers • Read/Write Reference Router tables – Longest Prefix Match – ARP – Destination IP Filter Building an Internet Router (P 33) Michaelmas 2009 37

Regression Test Examples • Reference Router – Packet Forwarding • regress/test_packet_forwarding – Longest Prefix Match • regress/test_lpm – Send and Receive • regress/test_send_rec Building an Internet Router (P 33) Michaelmas 2009 38

Creating a Regression Test • Useful functions: – nftest_regwrite(interface, addr, value) – nftest_regread(interface, addr) – nftest_send(interface, frame) – nftest_expect(interface, frame) – $pkt = NF 2: : IP_pkt->new(len => $length, DA => $DA, SA => $SA, ttl => $TTL, dst_ip => $dst_ip, src_ip => $src_ip); Building an Internet Router (P 33) Michaelmas 2009 39

Creating a Regression Test (2) • Your task: 1. Template files /root/NF 2/projects/bir_starter/regress/test_fwd_source. Port 0/run. pl 2. Implement your Perl verif tests Building an Internet Router (P 33) Michaelmas 2009 40

Running Regression Test • Run the command nf 2_regress_test. pl --project bir_starter Building an Internet Router (P 33) Michaelmas 2009 41