The Real Time Transport Protocol RTP Jonathan Rosenberg

The Real Time Transport Protocol (RTP) Jonathan Rosenberg Chief Scientist

CONNECTING THE WORLD WITH APPLICATIONS Talk Overview l RTP Functions l The Big Picture l RTP Services l RTCP Services l Scaling RTCP l Aggregation dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP: What is it? l Real Time Transport Protocol l RFC 1889 n product of avt working group n 1996 proposed standard n 2000 draft standard (new rfc) l What does it do n e 2 e transport of real time media n optimized for multicast (Mbone) n provides sequencing, timing, framing, loss detection n provides feedback on reception l What does it do (cont) n provides information on group members n provides data to correlate audio and video and other media l Features n Scales to huge multicast groups (millions) n Works with any codec l need payload format for each codec n Flexible quality dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP: What isn’t it? l Doesn’t guarantee quality of service l Not a specific protocol type n Does not run directly ontop of IP n doesn’t reserve network resources n Runs ontop of UDP n doesn’t guarantee no loss or n No fixed port number bounded delay n can work with Qo. S protocols (RSVP) l Doesn’t provide signaling n other protocols must be used to set up RTP (like SIP or H. 323) dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP Stack RTP RTCP UDP IP dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Big Picture: RTP, SDP and SIP Proxy C=IN IP 4 123. 1. 2. 3 m=audio RTP/AVP 1122 0 1 m=video RTP/AVP 1130 98 a=rtpmap: 98 h 263 SIP w/ SDP End User IP Network User RTP dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP Components: Data + Control l Data aka RTP n very confusing l Always on an even UDP port l Provides n sequencing n timing n framing n content labeling n User idenfitication dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL l Control = Real Time Control Protocol (RTCP) l Same address as data, but one higher port l Provides n reception quality n sender statistics n participant information (multicast - Mbone) n synchronization information www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Real Time Data Transport l Originator breaks stream into packets (segmentation) RTP Source n application layer framing (ALF)!!! l Packets sent; network may RTP lose, delay, reorder packets Packets l Must, at receiver: n reorder n recover n resegment RTP Sink n rescynchronize n clock synchronization! dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Transport System l Source l Sink n Digitize Audio from mike n Receive packets n Silence Suppression n Un-packetize n Echo cancellation n decompress n Compress Audio n comfort noise generation l G. 711: 64 kbps n reorder l G. 729: 8 kbps n recover loss l G. 723. 1: 5. 3/6. 3 kbps n jitter buffer n Packetize Audio in RTP n A/D conversion to speakers n Send dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Jitter Buffer l Packets delayed differently l Must play them out periodically pkts l Packets may arrive after designated playout time -> loss l Insert extra delay to compensate l May need to adapt this time amount dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP Packet Header +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | |. . | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP Header Fields l Version: 2 l P: indicates padding (for encryption) l X: extension bit l CSRC count: for mixers (later) l M: Marker Bit: indicates framing n audio codecs: first packet in talkspurt n video: last packet in frame dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL l Payload Type: indicates encoding n in RTP packet allows changes perpacket n Useful for: l adaptation l DTMF codec l silence codecs l SN: defines ordering of packets l Timestamp: when packet was generated l SSRC: identifier l CSRC: list of mixed users www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTP Timestamp l Tick units are dependent on codec n For speech: 125 microseconds (standard 8 khz sampling rate) n For video: 90 Kh. Z n For audio: 44. 1 Kh. Z (CD rate) l Gaps in TS, but not in SN mean silence l Video n Timestamp represents time at beginning of frame n Many packets may have same timestamp l Speech n Time per packet may vary n Depends on packetization: 20 - 100 ms typical l Initial value random for security dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Mixers and Translators l Mixer = Bridge n combines audio from many users into one stream MIXER n CSRC list indicates users being mixed l Translator n Converts from one format to another TRANSLATOR n For low speed access links dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTCP: Real Time Control Protocol l Sent by all participants n both senders and receivers (think multicast) l Same address as data, different port (one higher) l Sender Report n SSRC of sender n number of bytes sent n number of packets sent n wall time + RTP timestamp (for correlation) l Several RTCP Packet Types n Sender Report (SR): from senders n Receiver Report (RR): by receivers. Indicates reception quality n Source Descriptor (SDES): describes participant n receiver report data l SDES n CNAME of participant (unique) n SSRC of participant n name, address, email n BYE: sent when leaving dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Real Time Control Protocol l Receiver Reports l BYE packets n Block for each sender n reason for leaving n Per sender data contains n for mixers, SSRC of those users l SSRC of sender l fraction lost from sender l jitter seen fromsender l highest SN received l cumulative packets lost l DLSR: delay since last SR l LSR: time of last sender report dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL leaving www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Compound RTCP Packets l Actual UDP packet contains many RTCP packets l Always starts with SR or RR l Always contains SDES l May contain BYE UDP Hdr SR Header SR Data SSRC 1 RR Data SSRC 2 RR Data SDES Header Name Email dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS RTCP Announcement Interval l RTP used in large multicast groups (possibly thousands) l Everyone sends RTCP, even l Current RTCP Algorithm n Estimate group size L l listen for other RTCP packets l Build up table of SSRC/CNAME receivers l When to send RTCP? n Compute interval LC l Possibilities: n Every T seconds: RTCP bandwidth linear with group size. Bad! n Never: nahhh l C depends on desired RTCP bandwidth and average RTCP packet size n Randomize by 1/2 to 3/2 n Send packet, do it again n With N participants, every N*C seconds dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Algorithm Subtleties l Algorithm slightly different for senders n For senders, L is actually number of other senders n Gives much more bandwidth to data senders l Participants must measure l First packet has minimum interval of 2. 5 seconds n speed up knowing who you’re talking to l Reconsideration Algorithm n group sizes can be very dynamic avg. RTCP size on the fly to compute C l RTCP defines a minimum interval of 5 seconds dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Reconsideration Algorithm l Problem: n Many users join group at same time n Each thinks group size is 1 n Each sends packet right away! l Solution: Reconsideration n Don’t send packet! Check if group size has increased. If so, reschedule packet n That’s conditional reconsideration l Unconditional reconsideration n If group size hasn’t change, random number redrawn n Turns out this works much better… long story. l BYE Reconsideration n Stop everyone from sending BYE if they all leave group at same time l Reverse Reconsideration n If people leave, allow users to send packets more frequently right away n Always reschedule packet dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Payload Formats l Each codec needs a way to be encapsulated in RTP l RFC 1890 defines mechanisms for many common codecs n G. 711, G. 729, G. 723. 1, G. 722, etc. l Payload format defines n How to break frame into packets n extra fields needed below main RTP header n How to recover missing packets (sort of) n Some simple video l More complex codecs have their own payload format documents n MPEG n H. 263 and H. 261 dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Special Payload Formats l Redundant Encodings n Each packet contains two versions of voice l Current frame l Low bit rate encoding of previous frame n When packet is lost, wait for next one! n Introduces delay l FEC Payload Format (RFC 2733, issuing tomorrow) n Use parity codes (XOR) to protect packets n Take N packets, XOR them, get FEC packet n Send FEC packet n Can recover if any one of N packets is lost n Needs many codecs at source dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL www. dynamicsoft. com

CONNECTING THE WORLD WITH APPLICATIONS Special Formats l Aggregation n Mix many users together in a single packet n Useful for gateway to gateway communications n Can also reduce overheads dynamicsoft Inc. PROPRIATARY AND CONFIDENTIAL l Compressed RTP n For dialup links n Don’t send header, just send index n Far side uses index to retrieve header, and then increments certain fields www. dynamicsoft. com