Busses Interfaces CPUMEMORY CPUDEVICE Vocabulary Bus Shared communication

Busses & Interfaces CPU-MEMORY CPU-DEVICE

Vocabulary: Bus: Shared communication link, which uses a set of wires to connect multiple subsystems Can create a bottleneck since all commands/data pass through the bus Types of Transactions: Read from memory or device Write to memory or device Operation includes: Sending a command (e. g. , read/write) Sending an address (e. g. to read/write in memory) Sending or receiving data

Bus consists of multiple lines Bus contains transactions or separate lines for: Control (lines): Provides transaction command/response. For example: Read. Request: Proc Device: Indicates data lines contain address to read Data. Ready: Proc Device: Indicates that the data lines contain data to write Ack: Acknowledges a Data. Ready or Read. Request command Data (lines): Transmit data: 32, 64, 128+ lines: each line one bit Address (lines): Transmit address of memory or device to interface with Handshaking protocol: Sequence of transmissions completes the transaction. Standards define the protocol

Communications modes Synchronous: Includes a clock pin (= a metronome) to time input/output Protocol interface defined as a schedule Requires every device to operate at that clock rate Requires short bus length to avoid clock skew Used for Processor-Memory Buses Asynchronous: No clock pin is provided: internal clocks are used Requires a handshaking protocol to coordinate transmission Advantages: Variable speed, flexible bus length Used with I/O devices Serial: Transmits one bit at a time Parallel: Transmits multiple bits at a time Impacts speed: clock rate, number of pins (bits), optional address pins, etc. Currently high-speed serial point-to-point switched interconnections are popular

Bus Features Robustness: Hot pluggable: Can plug in/remove nodes while bus is operating Parity checking: Includes an extra bit for error detection Cyclic Redundancy Check: Superior: Includes extra bytes for error detection Arbitration scheme: Bus Contention required when multiple nodes connected to a bus Daisy chained: Extensions connect different nodes in single line. Nodes share request & grant signal and are linked according to priority. Self-selection: Addresses: 0001, 0010, 0100, 1000: First 1 bit transmitted wins: e. g. 0101

Types of buses Processor-Memory Bus: Transfers data between the processor and memory Includes: QPI Usually synchronous PC: Part of North Bridge I/O Bus: Transfers data between devices and processor. Includes: Fire. Wire, USB, SCSI, PCI Usually asynchronous PC: Part of South Bridge

Overview of various busses Thunderbolt USB 3. 1 PCI Express 4. 0 (Periph. Component Interconnect) Serial ATA 3. 0 SCSI (Small Comp. Sys. Interface) Internal/Extern Data al width to computer External – serial 2 input & 2 bus output External 2 Devices Internal – 2 per lane computer x up to 16 expansion bus lanes Internal - Disk External – electrical ribbon or optical connectors 4 Serial Max Cable Length Max Devices/ Controller Peak Bandwidth 10 Gbps 5 m 127 10 Gbps 528 MBps 1. 969 GBps (=1 x) 31. 51 GBps 1 m 12 m 16256 16 6 Gbps 320 MBps (Ultra) Up to 12 Gbps Fiber: 14 Gbps

“ CPU-I/O Bus Systems SCSI ARCHITECTURE MODEL (SAM): 2 LAYERS SCSI Parallel Interface (SPI) SCSI RDMA Protocol (SRP) Infini. Band Serial Attached SCSI (SAS) ” i. SCSI Fibre Channel (FCP) SSA SCSI -3 Serial Bus Protocol-2 Transport Protocol Internet Fibre Channel (FC-PH) SSA-PH 1 or SSA-PH 2 IEEE 1394 (PHY) Physical Interconnection

Small Computer System Interface: SCSI 1981: NCR & Shugart Assoc. Drive intelligence into the interface Today: SCSI-3 Used in 80% of enterprise-class storage systems Parallel SCSI: 10 -320 MBps 50 conductors: 8 for data; 11 for control Features: Daisy-chained: 16 -32 devices 2 Modes: Synchronous: bus reserved for a period of time Asynchronous: byte-by-byte data acknowledgments

Internet SCSI (i. SCSI) App OS SCSI Disk File System SCSI Driver SCSI Disk Controller SCIS Initiator System SCSI Initiator Subsystem TCP IP IP Internet Ethernet IP TCP i. SCSI Cmd Ethernet SCSI Data

SCSI Initiator Target (BSY Line low) Own Device ID ------------------------> (BSY high) Target Device ID------------------------> (SEL raised; BSY low) BSY raised (SEL low) <-------------Ready for CMD (C/D) + REQ) Command 1 --------------------------> ACK raised, C/D low C/D + REQ high Data 1 --------------------------------> ACK raised, C/D low <------------------------Cmd Complete + MSG All low Transfer Request: I want to transmit Arbitration: Highest priority request gets service Command State: Multiple cmds may occur Data State: Multiple data tx may occur

IEEE 1394: Apple Fire. Wire Speeds to 480 MBps or higher 6 Conductors: 4: data + control; 2: power Features: Daisy-chain 63 devices Length: 15 feet btwn devices Communicate to host, between devices Self-configuring: plug-and-play Serial bus Later: Thunderbolt: Combines data, video, audio, and 10 W power in one 10 Gbps connection Used for RAID arrays, video-capture, network interfaces

Serial Storage Architecture (SSA) Loop bus architecture: 4 -conductor cable: 2 twisted pair or 4 fibers Duplex: data sent in both directions 40 MBps in each direction Redundancy: Survives a single host failure

Fibre Channel Fiber optic bus system, exceedingly fast 3 modes: Switched (star), point-to-point, or loop; loop most common Switched (star) configuration uses hub or switch Hub forwards packets in all directions Switch forwards packets to destination only (intelligence) Disadvantages: Expensive High learning curve

Storage Area Network (SAN) Clients Features: Dedicated network built for storage Isolated from ordinary network traffic Redundant access – no bottlenecks Application Servers SCSI Arrays RAID 5 Fiber Channel Switches SCSI Switch

Peripheral Component Interconnect (PCI) Peripheral bus drives graphic displays, network interface controls, disk controllers Speeds to 66 MHz x 32 bit = 264 MBps Negotiates speed for slower devices For small home and high-performance systems Intel-sponsored, Non-SCSI PCI Express (PCIe) Point-to-point interconnect Supports higher data rates needed for video-on-demand, Gigabit Ethernet 1, 4, 6, 16 or 32 lanes PCI Express

Universal Serial Bus (USB) Serial Peripheral Interface Used on electronic consumer products: store data or rechargeable USB 3. 1 (2013): 10 Gbps in Superspeed: disk backups, video streaming Root hub: Adapter card in computer Multiport hubs: connects to root hub and many peripheral devices Developed) by USB Implementers Forum (industry standard: equipment manufacturers) USB A to B Cables

USB 4 -port hub Plug-and-Play Setup Publication of device driver software and host-device protocol : Host requests Device Descriptor info to device: device type, manufacturer’s code, product ID, USB spec number (e. g. , 2. 0) Host loads device driver If multiple configurations: host requests Configuration Descriptor Host assigns address to device Host & Device negotiate data transfer Data Transfer Modes Control transfer: Plug-and-play, setups for other transfers Isochronous transfer: Time-sensitive data transfer: music, video Interrupt transfer: Bursty data: mice, keyboard Bulk transfer: Bulk data: flash drives, cameras, scanners

My USB Internal Hub

My USB-Connected Web. Cam

My USB Mass Storage Device

Processor Memory I/O Bridge I/O Processor CPU-Memory Protocols QPI PCIE

Layered Architecture for Interconnect Protocols (These, QPI, and PCIe notes taken from: Computer Organization & Architecture, 9 th Ed. , William Stallings, Pearson Ed. 2013. ) Modern interfaces work mainly with switches or point-to-point interfaces. Transaction Defines and Exchanges Packets: Command Type: 1. Memory Read (0) / Write (1) E. g. , Read=10 2. I/O Read (0) /Write (1) E. g. , Write=21 3. Configuration: Command requests to I/O devices 4. Message: Error handling, interrupts, power mgmt 5. Completion: Command completed = 50 Sequence Number or Tag: Data: Address, Command, and/or Data Routing Optional: Routes packets through the system Source Address: Where packet is from Destination Address: Where packet is going to Reliable transmission and flow control Sequence Number: Orders packets, enables retransmissions, acknowledgments Cyclic Redundancy Check (CRC): Sophisticated checksum verifies accuracy Flow Control: Permission is given to send N packets. Electrical interface for transmission and reception: Data Lanes (or lines): Parallel transmission of bits Balanced Transmission: Return circuit: Tx 1 Rx 0; or Tx 0 Rx 1. Link Physical

Formatted packet Physical: Framing Link: Seq. # Start of Packet Routing: Src/Dest Transaction: Cmd(2) /Tag(2) Data Link Layer: Link: CRC E. g. , Disk read or write Flow Control: At initialization, credits are allocated Each ‘flit’ packet transmitted consumes one credit. When buffer processed at receiver, a credit is sent back. Error Control: Sender keeps copy of packet until it is acked When packet received with CRC error, bit(s) have been mutilated, NAK is returned Physical: Framing End of packet

Memory Architecture UMA: Uniform Memory Access: E. g. : All memory is shared on a bus for all processors NUMA: Non-Uniform Memory Access: Local memory and remote memory E. g. : Each processor has its own memory, which is not easily accessible to other processors

Quick. Path Interconnect (QPI)

Quick. Path Interconnect Details Characteristics include: Multiple direct connections: Each device may interface directly to other devices NUMA example shown Layered Protocol with packet data transfer Physical Layer: 20 data lanes (or lines) in each direction: 1 bit time= 20 bits Clock: Transmit in each direction Link Layer: Supports Flow Control & Error Control: Checks CRC and retransmits on error Routing Layer: Supports routing packets through other nodes QPI Protocol Layer: Sends data, but supports other features too: e. g. , cache coherency

Peripheral Component Interconnect Express (PCIe)

PCIe Transaction Layer Supports Transaction Types: System main memory and PCIe I/O devices: Some memory addresses map to I/O devices. Legacy PCI devices: Some addresses map to legacy I/O Configuration: Set or read device configuration via I/O device registers. Message: Control signals related to interrupts, error handling and power management

Conclusion Traditional Busses: Separate data, address, control lines Bus Contention: Determines who has control over the bus. Plug-and-Play feature possible Works with slower devices. Faster Interfaces: Layered protocol interface Supports Computer-Cache-Memory Point-to-point interface (only 2 devices) Includes error detection at link layer