RTP Multiple Stream Sessions and Simulcast draftwesterlundavtcoremultistreamandsimulcast00 Magnus

  • Slides: 20
Download presentation
RTP Multiple Stream Sessions and Simulcast draft-westerlund-avtcore-multistream-and-simulcast-00 Magnus Westerlund and Bo Burman Ericsson

RTP Multiple Stream Sessions and Simulcast draft-westerlund-avtcore-multistream-and-simulcast-00 Magnus Westerlund and Bo Burman Ericsson

IPR Disclosure › Telefonaktiebolaget LM Ericsson (publ)'s has made a Statement about IPR related

IPR Disclosure › Telefonaktiebolaget LM Ericsson (publ)'s has made a Statement about IPR related to this draft in https: //datatracker. ietf. org/ipr/1592/ draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 2

Outline › Problem Descriptions – – – A Target Scenario Simulcast Multiple Streams in

Outline › Problem Descriptions – – – A Target Scenario Simulcast Multiple Streams in Advanced RTP usages Bandwidth Signaling Codec Control / Optimization – – – Multiple Streams signaling Bandwidth SDP attribute Simulcast Grouping SRCNAME SDES item Codec Control › Problem Summary › Extensions › Way Forward – Architecture document – Extensions draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 3

A Target Scenario Active speaker … RTP Mixer Listener draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 |

A Target Scenario Active speaker … RTP Mixer Listener draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 4 Listener

Simulcast Listener When using scalable coding with inter-layer prediction, the broad arrows will be

Simulcast Listener When using scalable coding with inter-layer prediction, the broad arrows will be slightly smaller, but all red arrows will also have to be sent Listener Simulcast: sending multiple representations of the same source Listener Active speaker Listener RTP Mixer Listener draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 5 Listener

Simulcast and Scalable Encoding › Simulcast is both an alternative and complementary to Scalable

Simulcast and Scalable Encoding › Simulcast is both an alternative and complementary to Scalable Encoding › The trade-offs when it comes to efficiency are different – SVC encoding is more efficient in sender to mixer path – Simulcast is more efficient in mixer to receiver path – Combining scalable encoding with simulcast for best of both worlds › Simulcast is codec agnostic › Simulcast can be done for other purposes – Provide two different encodings for interoperability – Provide redundancy for robustness draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 6

Multiple Streams A sample client, both sending and receiving multiple video streams RTP Endpoint

Multiple Streams A sample client, both sending and receiving multiple video streams RTP Endpoint draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 7

Multi-Stream Layering ENC DEC SSRC 1 SSRC 2 … SSRC N RTP Session UDP

Multi-Stream Layering ENC DEC SSRC 1 SSRC 2 … SSRC N RTP Session UDP › An RTP Session can contain 1. . N SSRCs › An RTP Session is identified by a lower layer identifier, such as a UDP port or five tuples › A multimedia session contains one or more RTP sessions draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 8

Multi-Stream Simulcast Layering Scene Generator ENC DEC DEC SSRC 1 SSRC 2 … SSRC

Multi-Stream Simulcast Layering Scene Generator ENC DEC DEC SSRC 1 SSRC 2 … SSRC N RTP Session (Content: Main) RTP Session (Content: Alternative) UDP › A single source can be simulcasted as different versions – Same actual source in several sessions; don’t want to force new semantics into SSRC value › Several sources can be rendered at the same device draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 9

Multi-Stream Issues › More advanced use cases than point to point Vo. IP: –

Multi-Stream Issues › More advanced use cases than point to point Vo. IP: – Video conferencing – Telepresence – IPTV – Etc. › This can result in multiple media streams – Is the end-point capable of handling multiple simultaneous media streams of the same media type? › Legacy capabilities is likely one SSRC per direction – When should additional media streams be in the same RTP session, when in a new session? – When streams have relations, how to express that for: › Retransmission › Redundancy › Simulcast draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 10

Bandwidth Signaling › Current SDP Bandwidth signaling insufficient in handling: – Asymmetric bandwidth capabilities

Bandwidth Signaling › Current SDP Bandwidth signaling insufficient in handling: – Asymmetric bandwidth capabilities in the path – Asymmetric bandwidth usage inherent from application – When different Payload Types have different bandwidth ranges – When multi-stream applications use multiple streams in each direction – The allowed burstiness of media sources is not explicit VGA Endpoint HD pt=96 Alternate codecs Endpoint pt=8 draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 11

Codec Control / Optimization Active speaker Listener Simulcast Red, sent streams are not consumed

Codec Control / Optimization Active speaker Listener Simulcast Red, sent streams are not consumed by anyone (e. g. based on speaker activity) and should be paused and resumed on request Listener RTP Mixer Listener NOTE! Just as valid for scalable coding! draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 12 Listener

Problem Summary › We have a general RTP architecture clarity issue – We need

Problem Summary › We have a general RTP architecture clarity issue – We need to clarify multiple SSRCs in one RTP session – We need to discuss when appropriate to use multiple RTP sessions – We need to create common principles for streams that aren’t independent, but have common source. › We need to do this for all reasonable topologies › Multiple SSRCs in an RTP session appear to need signaling support to avoid legacy issues › Simulcast is good tool, we need signaling and RTP association mechanisms to make it work › Bandwidth configuration and capability declaration in asymmetric usages and encodings needs to be improved › Scalable Codecs and Simulcast needs additional Codec Control tools to optimize sessions draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 13

Proposed Extensions (1/4) › Multiple Streams Signaling – Separated directions › a=max-send-ssrc: 96 2

Proposed Extensions (1/4) › Multiple Streams Signaling – Separated directions › a=max-send-ssrc: 96 2 › a=max-recv-ssrc: 96 5 – Both payload specific and payload agnostic › a=max-recv-ssrc: 98 6 › a=max-recv-ssrc: 99 4 › a=max-recv-ssrc: * 8 draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 14

Proposed Extensions (2/4) › Bandwidth Signaling – b= line not possible to extend with

Proposed Extensions (2/4) › Bandwidth Signaling – b= line not possible to extend with sufficient new semantics – Per direction and payload type (also payload agnostic) – Per source › a=bw: recv pt=96 SMT: tb=64000: 320 › a=bw: recv pt=97 SMT: tb=12200: 128 – Entire media level aggregate › a=bw: send pt=* AMT: tb=384000: 512 – Allow for future needed semantics to be defined draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 15

Proposed Extensions (3/4) › Simulcast Grouping in SDP – Different semantics between directions –

Proposed Extensions (3/4) › Simulcast Grouping in SDP – Different semantics between directions – a=group: SCS 1 2 3 … (Simul. Cast Send intention) – a=group: SCR 4 5 … (Simul. Cast Receive capability / acknowledge) › Simulcast Source Identification in RTP – SDES CNAME is defined as unique per endpoint, not per source – New SDES SRCNAME unique per actual media source › Indicate which streams are alternative encodings to each other draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 16

Proposed Extensions (4/4) › Codec Control extensions is forthcomming draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 |

Proposed Extensions (4/4) › Codec Control extensions is forthcomming draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 17

Anticipated Document Split Current RTP Session and Multistream Architecture Multistream Signalling Simulcast Grouping draft-westerlund-avtcore-multistream-and-simulcast-00

Anticipated Document Split Current RTP Session and Multistream Architecture Multistream Signalling Simulcast Grouping draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 18 Bandwidth Codec Control

Proposal for Going Forward › That AVTCORE takes on the general Architecture questions: –

Proposal for Going Forward › That AVTCORE takes on the general Architecture questions: – Make it clear when appropriate to use multiple streams within an RTP session – How should one use RTP sessions and SSRCs when having alternative, complementary or redundant streams › That the various extensions are submitted to the appropriate WG as individual pieces for progressing: – AVTCORE: › Architecture › Multi-stream Signaling – AVTEXT: › Simulcast Group Signaling › Codec Control – MMUSIC: › Bandwidth Signaling draft-westerlund-avtcore-multistream-and-simulcast-00 | IETF 81 | 2011 -07 -14 | Page 19