RealTime Transport Protocol RTP Tung Dao Manh Future

Real-Time Transport Protocol (RTP) Tung Dao Manh Future Internet Class, 2007. 04. 15 Pohang University of Science and Technology (POSTECH) Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 1

Agenda Introduction Motivation RTP outline Fundamental design philosophies of RTP Application-level framing The end-to-end principle Flexibility Standard elements of RTP Specification RTP Profile RTP Payload Format RTP Packet Format Potential further development of RTP Related Protocol RTCP Conclusion POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 2

Introduction Motivation l Requirements for delivery of media stream § Examples of real-time applications: • Videoconferencing • Vo. IP, IP telephone system • Game online… § Real-time data: interactive audio and video § Receivers: playing out immediately and synchronously, rather than playing back § Requirements for transport protocols: § Predictable variation in network transit time § Reliable delivery of all packets POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

Introduction Motivation l Requirements drive the choice of transport protocols § TCP/IP § It favors reliability over timeliness, but these applications require timely delivery. Retransmissions can lead to high delay and cause delay jitter § Does not support multicast § Congestion control mechanism not suitable for audio-video (AV) media § UDP/IP § A UDP/IP-based should be suitable, provided that the variation in transit time of the network can be characterized and loss rates are acceptable. But it is: § No defined technique for synchronizing § Streams from different servers may collide § A feedback channel must be defined for quality control POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

Introduction Motivation l The standard real-time transport protocol was developed by the Audio-Video Transport Working Group of the IETF and first published in 1996 as RFC 1889 – A solution to these described problems. POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

Introduction Real-time data transport at glance • • • POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea Encoded frames are produced They are assigned a timestamp and a sequence number Then, they are load into RTP packet and ready for transmission The sender also generates periodic status reports in the form of RTCP packets to participants The participants also send quality feedback to the sender. Collect packets from network, and insert them into a per-sender input queue. Packets then are passed to a channelcoding routine for loss correct. Next, they are put into a playout-buffer, and any variation in interpacket timing caused by the network has been smoothed. They are then grouped to form complete frames that are decoded to play out. 8

Introduction RTP Outline l RTP: Defines a standardized packet format for delivering audio and video over the internet. l Internet standard for real-time data l Interactive and streamed audio and video l Designed for multi-user multimedia conferencing l Provides end-to-end transport functions for real-time applications l Delay-oriented rather than loss-oriented (such as TCP) POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP Fundamental design philosophies of RTP l Application-Level Framing l A transport protocol should accept data in application-meaningful units (ADUs) l A transport protocol should expose the details of the data delivery as much as possible l The end-to-end principle l The system can pass responsibility for the correct delivery of data along that with data, meaning it relies on the lower protocols. l Intelligence is at the endpoints, not within the network (smart, networkaware endpoints and a dumb network) l Achieving flexibility l Provides a unifying framework for real-time audio/video transport, satisfying most application directly. POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP l l Standard elements of RTP Framework RTP specifications RTP profiles RTP payload formats Optional elements POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

Introduction RTP Outline l A session consists of an RTP/RTCP pair of channels, containing two closely linked parts: Data + Control. l RTP to transport real-time data l RTCP: RTP control protocol l Qo. S monitoring and feedback l Session control l Protocol architecture: usually works over UDP/IP Media Application RTCP RTP UDP IP POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

Introduction RTP Outline POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

Introduction RTP Outline l RTP Issues: The protocol itself does not l provide mechanisms to ensure timely delivery l Relies on lower-layer protocols l prevent out-of-order delivery of packets l give any Quality of Service (Qo. S) Guarantee. l RTP Timestamp (TS) and Sequence Number (SN) l TS is used to order packets in correct timing order l SN detect packet loss POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP Features l Multicasting l Payload type identification l Time stamping l Enable timing recovery l Sequencing l Loss detection l Delivery monitoring POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP Packet Format 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | (0~15 items). . | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Header extension (optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload (real time data) | Padding (size Padding size | | X 8 bits) (8 bits) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ • Version (V): • Payload Type (PT) • Padding (P) • Sequence Number • Extension (X) • Timestamp • CSRC Count (CC) • Synchronization Source (SSRC) • Marker (M) • Contributing Source (CSRC) POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP • • • RTP header Version (V, 2 bits): indicate the version of the protocol (Current version is 2) Padding (P, 1 bit): if set, last byte of payload is padding size Extension (X, 1 bit): if set, variable-size header extension exists CSRC Count (CC, 4 bit): number of SCRC identifiers Marker (M, 1 bit): defined in profile, mark significant event Payload type (7 bits): audio/video encoding scheme Sequence number: random initial value, increase by one for each RTP packet; for loss detection and sequence restoration SSRC: identify source; chosen randomly and locally; collision needs to be resolved CSRC list: identifiers of contributing sources, inserted by mixer POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP header: SSRC and CSRC l All packets from a given synchronizing source (with a given SSRC identifier) will use the same timing and sequence number space to allow receivers to recreate the packet sequence l A mixer receives RTP packets from multiple sources, combines packets, makes timing adjustments, and forwards new RTP packets with a new timing sequence l All packets in the new sequence will have mixer SSRC as their synchronization source l The mixer inserts in each RTP packet header a CSRC list of the sources that contributed to this combined stream POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP header: Timestamp l l l Reflects sampling instance of the first byte in payload Clock frequency depends on data type; specified in profile Random initial value Example: CBR audio, clock increment by 1 for each sample Consecutive RTP packets may have same timestamp (logically generate at same instant): video packets that belong to the same frame l Timestamps of consecutive RTP packets may not increase monotonically if the data is not transmitted in the order in which it was sampled: MPEG interpolated video frames POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP header: Payload types l Some examples (RFC 1890) Payload type Encoding name Audio/Video (A/V) Clock rate 0 PCMU (mu-law G. 711) A 8000 8 PCMA (A-law G. 711) A 8000 26 JPEG V 90000 32 MPV (MPEG-I and MPEG II) V 90000 33 MP 2 T (MPEG- II AV transport stream) 90000 POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP RTP-based networking application l RTP implementation is expected to be integrated into the application rather than as a separate module. l The use of RTP for a particular application requires other documents l Profile specification documents defines sets of payload type codes, and their mapping to payload formats l Payload format specification documents define how to carry a specific encoding l Example: MPEG 2 video or ADPCM audio l RFC 1890: payload types; RFC 2250: MPEG POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP RTP-based networking application l RTP is part of the application and lies above the UDP socket Application RTP Socket UDP IP Data Link Physical POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP RTP-based networking application l RTP can be viewed as a sub-layer of the transport layer Application RTP UDP Transport IP Data Link Physical POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Introduction – RTP Control Protocol l Scenario l l l Receivers send reports Report contains number of packets lost at receiver, inter-arrival jitter, etc. This allows senders to adjust date rate Jitter is an early indicator of congestion Senders also send reports POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Introduction - Goals l The control protocol, RTCP, provides for periodic reporting of reception quality, participant identification and other source description information, notification on changes in session membership, and the information needed synchronize media streams. Canonical Name Identify and keep track of participant Qo. S feedback RTCP Minimal session control information POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Packet Types Bye: End of session APP: Application specifics SDES: Source Description RR: Receive report SR: Sender report RTCP packet types POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Compound RTCP Packet POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Compound RTCP Packet POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Sender report (SR) l SSRC: identifies source of data l Sender information blocks: l NTP timestamp: absolute time identifying when report was sent l RTP timestamp: time when packet is sent according to the clock used to send RTP data packet timestamps; used for synchronization l Sender’s packet count: Total number of packets sent since the start of session l Sender’s octet count: Total number of bytes sent since the start of session l Multiple receiver report blocks, one for each source from which this host receives packets POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Receiver report (RR) l SSRC: identifies source whose data this block is about (who’s being reported on) l Loss fraction: fraction of packets lost since last report was sent l Cumulative number of packets lost: long-term loss since the beginning of reception l Highest sequence number received: compare losses, disconnect l Interarrival jitter: smoothed interpacket distortion l LSR: The NTP timestamp of the last sender report received from the source l DLSR: Delay between receiving the last SR from this source and sending this RR POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Round-trip delay estimation POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Intermedia synchronization POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP SDES Packet Type l Source description (SDES): provide participant identification and supplementary details, such as location, emai address, etc. l The standard items are: CNAME, EMAIL, PHONE, LOC, TOOL, NOTE, and PRIV. l Constant for a user, unique among all users l Providing binding across multiple medias sent by a user l Example: user@postech. ac. kr POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTCP Analyzing sender and receiver reports l Sender may modify its transmissions based on the feedback l Receivers can determine whether problems are local, regional or global l Network managers may use profile-independent monitors that receive only the RTCP packets and not the corresponding RTP data packets to evaluate the performance od their networks for multicast distribution. POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP/RTCP Other issues l Collision detection and resolution l Two sources use the same SSRC l Loop detection l Security l Header compression – RFC 2508 POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP/RTCP Program Applications using RTP Capability Protocols AOL instance messenger Video, file transfer, PC to phone, phone to PC SIP, RTP i. Soft. Phone Address Book Integration, Daylite CRM Integration, Call Recording, IM, Conferencing, Mulitple Providers, Simple Setup, Call transfer, 5 lines SIP, STUN, RTP Windows Live Messenger Video, voice, chat, text messaging, PC to Phone SIP, RTP Yahoo! Messenger Video, file transfer, PC to phone, phone to PC SIP (using TLS) and RTP (media) Sight. Speed video, voicemail, phone in, phone out, multiparty calling, conference recording, text messaging, NAT traversal, video mail SIP, RTP, Proprietary P 2 P protocol POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8

RTP/RTCP Questions 감사합니다 & Question? POSTECH Copyright © 2007 SE Lab. Dept. of CSE POSTECH, R. O. Korea 8