Fibre Channel Chapter 9 1 Fibre Channel Introduction

Fibre Channel – Chapter 9 1

Fibre Channel Introduction n Originally developed for mainframe & supercomputing environments to connect together high speed clusters & storage Development began in 1988 under the auspices of the ANSI T 11 committee (device level interfaces) and culminated in the approval of the ANSI standard in 1994 Besides its use as a very high bandwidth I/O channel technology, there is increasing interest in Fibre Channel as a LAN technology because of its high speed and unique combination of channel & network oriented properties: – – – Data-type qualifiers for routing data into specific interface buffers Link-level constructs designed to support individual I/O operations Support for existing I/O interface specifications (SCSI, HIPPI, etc. ) Full multiplexing capabilities Peer-to-peer connectivity between any two ports in a FC network Ability to internetwork with other LAN, WAN, & I/O technologies 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 2

Fibre Channel Introduction n Comparsion of Fibre Channel with Gigabit Ethernet and ATM [Table 9. 1] 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 3

Fibre Channel Architecture n n Designed to provide a common, efficient, high-speed transport to a wide variety of devices through a single port type Requirements outlined by the Fibre Channel Association: – – – – – n Full-duplex links over a fiber pair (one transmit/one receive) Bi-directional performance up to 3. 2 -Gbps on a single link Support over distances up to 10 kilometers Small connectors for high density applications High-capacity utilization with distance insensitivity Greater connectivity than existing multi-drop channels Broad availability at reasonable cost Support for multiple cost/performance levels, from PCs to clusters Ability to carry multiple protocols and command sets The best way to meet such demanding requirements was to develop a transport mechanism based on simple point-to-point links & a switching network 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 4

Fibre Channel Terminology n Fibre Channel, having a different heritage than other LAN/WAN technologies, has different terminology [Table 9. 2] – – – Dedicated Connection: A circuit guaranteed and retained by the fabric for two specified N_Ports Exchange: The basic mechanism that transfers information, consisting of one or more related non-concurrent sequences in one or both directions Fabric: The entity that interconnects various N_Ports attached to it and handle the routing of frames Intermix: A mode of service that reserves the full FC capacity for a dedicated (Class 1) connection but allows the transport of additional connectionless data if space is available Node: A collection of one or more N_Ports Operation: A set of one or more, possibly concurrent, exchanges that is associated with a logical construct above the FC-2 layer 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 5

Fibre Channel Terminology (continued) n Fibre Channel, having a different heritage than other LAN/WAN technologies, has different terminology [Table 9. 2] – – – Dedicated Connection: A circuit guaranteed and retained by the fabric for two Originator: The logical function associated with an N_Ports that initiates an exchange Port: The hardware entity within a node that performs data communications over a FC link Responder: The logical function in a N_Port responsible for supporting an exchange initiated by an originator Sequence: A set of one or more data frames with a common sequence ID transmitted unidirectionally from one N_Port to another N_Port, with a corresponding response, if applicable, transmitted in response to each data frame 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 6

Fibre Channel Terminology n Fibre Channel Elements – – The key elements of a FC network are the end devices called nodes and the collection of switching elements called the fabric Communication between nodes across a FC network consists of transmission of frames across the point-to-point links & fabric Each node has one or more N_Ports for connection to the fabric Nodes connect to F_Ports on the fabric via bi-directional point-to -point links n n n – Fabrics can be a single switch or a general collection of switching elements Frames may be buffered within the fabric, making it possible for nodes to connect to the fabric at different data rates The fabric is a switched architecture, not a shared access medium, so no MAC issues are encountered and no MAC sublayer is necessary The FC network scales easily in terms of ports, data rate, and distance covered and through its layered protocol architecture interworks with existing LAN and I/O protocols 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 7

Fibre Channel Terminology n Basic Fibre Channel Architectural Diagram 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 8

Fibre Channel Example Architecture 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 9

Fibre Channel Protocol Specifications n Fibre Channel Protocol Architecture – The Fibre Channel standard reference model is organized into five levels [Figure 9. 3 and Table 9. 3] n n – – – These are not ‘levels’ in the strict sense of the OSI model but are instead functional groupings of services and/or definitions The standard does not dictate actual implementations, relationships between the levels, or the specific interfaces between levels Levels FC-0, FC-1, and FC-2 are defined together in a standard called the Fibre Channel Physical and Signaling Interface (FCPH) No final standard has been issued for FC-3 A number of standards have been developed at FC-4 specifying how Fibre Channel interfaces to existing LAN and I/O technologies 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 10

Fibre Channel Protocol Specifications n Fibre Channel Protocol Architecture (continued) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 11

Fibre Channel Protocol Specifications n Fibre Channel Protocol Architecture (continued) – Details on the FC-0 level n – A variety of physical media and data rates are allowed: – Data rates: 100 -Mbps to 3. 2 -Gbps – Media: fiber optic, coaxial cable, and STP – Distance: 50 m to 10 km depending on data rate and media The FC-1 level uses a 8 B/10 B encoding scheme in which 8 bits of data from the FC-2 level are encoded into a 10 bit binary symbol 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 12

Fibre Channel Protocol Specifications n Fibre Channel Protocol Architecture (continued) – The FC-2 level is responsible for the transmission of data between N_Ports, which requires the following: n n n – Addressing of N_Ports Permissible topologies of the fabric Classes of service Segmentation and reassembly of frames as well as higher level grouping of frames (sequences and exchanges) Sequencing, flow control, and error control The FC-3 level provides a common set of services across multiple N_Ports n n n 635. 412. 71 Spring 05 Striping: the process of using multiple ports to transmit a single data unit in parallel Hunt groups: allows a connection to be established to any available N_Port in the group Multicast (and broadcast) Class #5: Token Ring LANs & Fibre Channel 13

Fibre Channel Protocol Specifications n Fibre Channel Protocol Architecture (continued) – The FC-4 level defines how other protocols interoperate with Fibre Channel (specifically FC-PH) n n n 635. 412. 71 Spring 05 SCSI – a common device interface standard for computer peripherals HIPPI – a high speed I/O channel used in mainframe and supercomputing environments IEEE 802 – how IEEE 802 MAC frames map to Fibre Channel frames ATM IP – how to map packets into Fibre Channel frames (RFC 2625) Class #5: Token Ring LANs & Fibre Channel 14

Fibre Channel Physical Media and Topologies n n A major strength of Fibre Channel is that it provides a range of options for the physical medium, the data rate on that medium, and the topology of the network Transmission Media – A special shorthand nomenclature has been developed for FC media – it basically consists of the following: n n – Speed-Medium-Transmitter-Distance FC-0 options are listed in Figure 9. 4 Allowable Media Types n n n 635. 412. 71 Spring 05 Fiber Optic: both SM and both 50 m and 62. 5 m MM Coaxial Cable: three types of 75 ohm cable are specified, a thick RG-6/U, a thinner RG-59/U, and a miniature coax cable 0. 1 inches in diameter Shielded Twisted Pair: two types of 150 ohm cables are specified for use over short distances at data rates up to 200 Mbps: EIA-568 Type 1 STP: (two shielded twisted pair) or EIA 568 Type 2 STP (four pair STP) Class #5: Token Ring LANs & Fibre Channel 15

Fibre Channel Physical Media and Topologies n Topologies – – The most general FC topology is the fabric (switched) topology Four basic topologies [Figure 9. 5] are available in Fibre Channel: point-to-point, fabric, arbitrated loop (no hub), and arbitrated loop with hub n n Point-to-point connects two end nodes with no switches or routing The fabric topology can contain an arbitrary number of switches, some connecting to nodes and others that just provide transport between other switches – – – The fabric topology allows for easy scalability In the fabric topology the overhead on nodes is minimized; they are only responsible for managing the point-to-point link to their local switch Each port requires a unique address to allow frames to be delivered to the proper destination 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 16

Fibre Channel Physical Media and Topologies n Topologies (continued) – The arbitrated loop topology allows up to 126 nodes to be connected in a simple, low-cost loop n n n – – The ports on the loop are a special kind called NL_Ports because they must perform special functions associated with loop management Operation is roughly equivalent to other token ring protocols There is a token acquisition protocol controlling loop access The fabric & loop topologies can be connected as long as one node can act as both an arbitrated loop & a fabric node that participates in routing decisions on the fabric The topology of a given FC network is discovered automatically as part of network initialization 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 17

Fibre Channel Physical Media and Topologies n Fibre Channel Topologies (continued) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 18

Fibre Channel Framing & Classes of Service n Framing Protocol – – – The FC-2 layer defines the rules for the transfer of frames between nodes, comparable to the OSI data link layer FC-2 specifies the types of frames, procedures for the exchange of frames, frame formats, flow control, and classes of service Classes of Service n n FC-2 defines multiple classes of service; these classes are determined by the way communication is established between two ports and their flow control and error control capabilities Five classes of service are currently defined: – – – 635. 412. 71 Spring 05 Class Class 1: 2: 3: 4: 6: Acknowledged Connection-oriented service Acknowledged Connectionless service Unacknowledged Connectionless service Fractional Bandwidth Connection-oriented service Unidirectional Connection service Class #5: Token Ring LANs & Fibre Channel 19

Fibre Channel Framing & Classes of Service n FC-2 Classes of Service – Class 1 Service (Acknowledged Connection-oriented service) n n 635. 412. 71 Spring 05 Provides a dedicated path through the fabric which behaves to the end nodes like a point-to-point link Also provides a guaranteed data rate with sequenced delivery of frames The end node requests the setup of a Class 1 service connection using a special start-of-frame delimiter (SOFc 1) Class 1 service is advantageous for long constant bandwidth transfers of data (e. g. - streaming backups over a network) Class #5: Token Ring LANs & Fibre Channel 20

Fibre Channel Framing & Classes of Service n FC-2 Classes of Service (continued) – Class 2 Service (Acknowledged Connectionless service) n n n 635. 412. 71 Spring 05 Provides an acknowledged data transmission service without the overhead of setting up a connection through the fabric Acknowledgements frames are returned by the receiving port, if a delivery cannot be made due to congestion a busy frame is returned This is not the case with frames that cannot be delivered due to frame errors Sequenced delivery is not guaranteed; frames can take different paths through the fabric if possible Multiplexing of frames from different sources and/or destinations is allowed Class 2 service is good for Storage Area Networks (SANs) Class #5: Token Ring LANs & Fibre Channel 21

Fibre Channel Framing & Classes of Service n FC-2 Classes of Service (continued) – Class 3 Service n n n – Provides a basic datagram service with no connection setup No guaranteed nor acknowledged delivery Good for short bursts of data or delivery of multicast/broadcast data Class 4 Service n n 635. 412. 71 Spring 05 (Fractional Bandwidth Connection-oriented service) Provides a service similar to Class 1 but also provides Quality of Service (Qo. S) guarantees and reservations Allows the specification of guaranteed bandwidth & bounded latency Qo. S parameters established separately for each direction Good for time-critical & real-time applications like videoconferencing Class 6 Service n (Unacknowledged Connectionless service) (Unidirectional Connection service) Provides the reliable unicast delivery found in Class 1 but also supports reliable multicast and preemption Good for video streaming and broadcasting Class #5: Token Ring LANs & Fibre Channel 22

Fibre Channel Frames, sequences, and exchanges n There is much more to the FC-2 layer than frames & classes of service; it defines a set of functional building blocks for higher layer services – – – Also defines a number of protocols used to implement services at a port Typical protocols are creating or terminating a connection, transferring data, etc. Protocols consist of an exchange of information between N_Ports, which in turn consists of sequences, and sequences a composed of a related set of frames 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 23

Fibre Channel Frames, sequences, and exchanges 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 24

Fibre Channel Frames, sequences, and exchanges (continued) n There are two general types of frames: data and control – The three types of data frames are used to transfer higher level information between N_Ports n n n – FC-4 Device Data: used to transfer higher-layer data units from protocols specified in FC-4 standards (IP, SCSI, etc. ) FC-4 Video Data: used to transmit streamed video between buffers without an intermediate storage Link Data: used to support higher level control information between N_Ports There are currently three types of link control frames defined: n n n 635. 412. 71 Spring 05 Link Continue: functions as an acknowledgement in Fibre Channel sliding-window based data transfer Link Response: used as a negative acknowledgement in FC slidingwindow based data transfer Link Command: A reset command used to reinitialize the slidingwindow based transfer mechanism Class #5: Token Ring LANs & Fibre Channel 25

Fibre Channel Frames, sequences, and exchanges (continued) n Sequences – – – With Fibre Channel a maximum frame size is imposed at the FC-2 layer but is transparent to higher layers Higher layers set down chunks of data to FC-2, which may need to break them up into a sequence of frames The sequence of data frames needed to carry a single higher -layer chunk of data may also be accompanied by one or more link control frames for acknowledgement FC-2 provides the segmentation and reassembly that supports the transmission of sequences as well as error control Errors in a frame that belongs to a sequence causes the retransmission of that whole sequence (and any others transmitted after it – go back N ARQ) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 26

Fibre Channel Frames, sequences, and exchanges (continued) n Exchanges – – Exchanges are mechanisms for organizing multiple sequences into a higher-level construct to allow easier interfacing to applications Examples of exchanges are SCSI disk operations like a read or write Can involve either a unidirectional or bi-directional transfer of sequences Within a given exchange, only a single sequence can be active (though sequences from different exchanges can be simultaneously active) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 27

Fibre Channel Frames, sequences, and exchanges (continued) n Protocols – – An exchange is tied to a protocol that provides a specific service for higher levels Some common protocols that may be used by any higher application: n n n 635. 412. 71 Spring 05 Fabric Login: executed upon initialization of an N_Port, requires the exchange of the N_Port address, classes of service supported, and flow-control parameters N_Port Login: the exchange of service parameters between a pair of N_Ports before data exchange (buffer space, service classes supported, etc. ) N_Port Logout: the termination of a connection between a pair of N_Ports Class #5: Token Ring LANs & Fibre Channel 28

Fibre Channel Framing & Classes of Service n Two levels of credit are in use with Class 1 & 2 services -- end-toend and node-toswitch Class 4 may have the same flow control as Classes 1 and 2; can’t find a good answer because most current equipment only supports class 2 & 3 Flow Control – – – Fibre Channel provides a sophisticated set of flow control mechanisms at two ‘levels’: end-to-end and buffer-to-buffer The key to the FC flow control mechanisms is the concept of credit: credit is negotiated at login and denotes the number of unacknowledged frames allowed at any time End-to-End Flow Control n n 635. 412. 71 Spring 05 This type of flow control paces the flow of frames between N_Ports Requires acknowledgements to operate, so end-to-end flow control can be used only with Class 1 and Class 2 services Class #5: Token Ring LANs & Fibre Channel 29

Fibre Channel Flow Control (continued) n End-to-End Flow Control – Three types of acknowledgements are possible in a Class 1 or Class 2 service n n n ACK_1: acknowledges one data frame & decrements the credit count by 1 ACK_N: acknowledges N data frames & decrements the credit count by N ACK_0: acknowledges a whole sequence, decrementing the credit count by the number of frames in the sequence 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 30

Fibre Channel Flow Control (continued) n End-to-End Flow Control (continued) – – Acknowledgement types cannot be mixed; if ACK_1 is initially used for a Class 1 connection than it must be used for the duration of the connection Busy and Reject control frames are used for flow control n n 635. 412. 71 Spring 05 The F_BSY frame indicates the fabric is busy and cannot deliver a frame The P_BSY frame indicates the destination port is busy and cannot accept a frame; the sender will try a predefined number of times to retransmit the frame With the Reject (F_RJT and P_RJT) frames, delivery of the data frame is being denied (for some reason other than congestion) When a frame belonging to a sequence is rejected the whole sequence must be retransmitted Class #5: Token Ring LANs & Fibre Channel 31

Fibre Channel Flow Control (continued) n Buffer-to-buffer Flow Control n n n This is flow control across a pair of ports connected by a point-to-point link, assuring that buffers are available in the ports at either end of the link This mechanism is also applicable to all classes of service (including Class 3 datagram service) A single type of control signal, the R_RDY frame, is used for buffer-to-buffer flow control – – 635. 412. 71 Spring 05 As a data frame is transmitted across the link, the sender increments its credit count for the link At the receiving port the data frame is buffered as received As soon as the data frame is switched to another port’s buffer on the switch, the receiving port sends back the R_RDY frame to the sending port When the sending port receives the R_RDY frame it decrements the credit count, opening its send window by one frame Class #5: Token Ring LANs & Fibre Channel 32

Fibre Channel Framing & Classes of Service n Frame Format [Figure 9. 10] – The Fibre Channel Frame contains five general fields: n Start Delimiter n Frame Header n Data n Cyclic Redundancy Check (CRC) n End Delimiter 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 33

Fibre Channel Framing & Classes of Service n Frame Format - Start of Frame Delimiter – – – The start of Frame Delimiter includes a four byte set of nondata symbols denoting the start of a frame and allowing synchronization The SOF delimiter comes in several varieties, each of which will specify the frame’s type and class of service Examples are SOF Class 1 connection (SOFc 1), SOF normal (for data frames), and SOF fabric (for control frames in the fabric) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 34

Fibre Channel Framing & Classes of Service n Frame Format - FC- 2 Frame Header – Contains the control data required at this level; consists of the following fields: n n Routing control: contains two subfields, one that denotes the type of frame (device data, link control, etc. ) and the type of data within the frame Destination Identifier: destination N_Port or F_Port – – – 635. 412. 71 Spring 05 FC uses two levels of addressing: a globally unique identifier (world wide port/node names) & a lower level port identifier • World wide/port name is used by higher layers and for network management • Port identifier is the 3 -byte that is used for frame routing that consists of three parts: domain, area, and port The hierarchical addressing structure facilitates routing and management of the fabric A mechanism for mapping between the two addresses is necessary Class #5: Token Ring LANs & Fibre Channel 35

Fibre Channel Framing & Classes of Service n Frame Format - FC- 2 Frame Header – Contains the control data required at this level; consists of the following fields: n n Source Identifier: source N_Port or F_Port Type: if the routing control field specifies an FC-4 frame, then this field specifies the payload protocol (SCSI, IP, etc. ) – n Frame control: contains control information relating to frame content – n n This field and the Route control field allow the destination N_Port to deliver the data to the correct higher layer ‘user’ Is frame a retransmission? Is frame part of a sequence? Sequence ID: unique identifier for a sequence used for all frames belonging to it Data Field control: specifies which, if any, of four optional headers are present 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 36

Fibre Channel Framing & Classes of Service n Frame Format - Frame Header (continued) – Contains the control data required at this level; consists of the following fields: n n Sequence count: A unique number assigned sequentially to each frame in a sequence (for flow control and proper reassembly of frames within a sequence) Originator Exchange Identifier: a unique identifier assigned to the higher layer initiator of an exchange Responder Exchange Identifier: a unique identifier assigned to the higher layer destination of an exchange Parameter: used in different ways for link control and data frames – – 635. 412. 71 Spring 05 Link control frames carry information specific to the control function in this field Data frames may carry an address meaningful to the upper layer protocol Class #5: Token Ring LANs & Fibre Channel 37

Fibre Channel Framing & Classes of Service n Frame Format - Data Field – – Contains user data in a multiple of four bytes chunks up to a maximum of 2112 bytes Can also include one or more optional headers whose presence is denoted in the Data Field control field: n n Expiration Security optional header: can carry an expiration date for the frame and well as other security data over and above the FC-PH standard Optional Network Header: may be used by a bridge or gateway node interfacing to an external network to allow tunneling (includes 8 bit source and destination network addresses) Optional Association Header: may help specify an upper layer process (or group of processes) associated with an exchange Optional Device Header: if used the format is specified by the upper layer protocol used with the frame 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 38

Fibre Channel Framing & Classes of Service n Frame Format - CRC & End Delimeter – – CRC field: the error detection algorithm is the same 32 bit CRC used with FDDI and IEEE 802 End of Frame Delimiter n n n A four byte field denoting the end of the frame The EOF field may be modified by a switch in the fabric if it finds an error in the frame or some other condition that invalidates the frame There are three different EOF delimiters for valid frames: – – – 635. 412. 71 Spring 05 EOFt denotes the end of a valid sequence EOFdt is used with Class 1 service to indicate that the frame is the last frame on the logical connection (i. e. – the connection is being terminated) EOFn is used to denote successful transmission of frames not covered by the first two Class #5: Token Ring LANs & Fibre Channel 39

Fibre Channel Examples of Equipment n Fibre Channel Equipment Manufacturers – High-end (“Director-Class”) Switches n Brocade Silkworm 2400 (http: //www. brocade. com/products/directors/silkworm_24000/index. jsp) n Mc. Data Intrepid 6140 (http: //www. mcdata. com/products/hardware/director/6140. html) – Low-end (“Edge”) Switches n EMC DS-16 B 3 (http: //www. emc. com/pdf/products/connectrix_DS_16 B 2. pdf) n Cisco MDS 9120 (http: //www. cisco. com/en/US/products/ps 5993/index. html) – Host-Bus Adapters (HBA) n HP Storageworks FCA-2408 2 Gbps PCI-X (http: //h 18006. www 1. hp. com/products/storageworks/fca 2408/index. html) n Qlogic QLA 2200 L 1 Gbps PCI (http: //www. qlogic. com/support/product_resources. asp? id=118) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 40

IEEE 802. 3 Family of LAN Protocols Homework & Reading n Homework #4 - Due in four weeks (3/22) – – – n The idea of using Ethernet as a service provider technology is very attractive, but it lacks much of the functionality needed in that environment. Research a technology called Resilient Packet Ring (RPR) and write 1 -1. 5 pages on what its goals are and what functionality it provides. Fibre Channel continues to evolve as a networking technology: research and write 1 -1. 5 pages on two different enhancements are currently being developed (e. g. – higher speeds, new higher layer mappings, etc. ) Redo OPNet Lab #1 using a 16 -Mbps Token Ring instead of ethernet; answer all questions except #4. Reading – – This week’s material: Stallings chapters 8 and 9 Next week: SONET, ATM, & ATM LANs (chapter 11) 635. 412. 71 Spring 05 Class #5: Token Ring LANs & Fibre Channel 41