Web RTC 101 by Shuai Zhao Outline Introduction

Web. RTC 101 by Shuai Zhao

Outline • Introduction • Signaling Server Support • Web. RTC APIs. • How to use it? • Demo

History • Starts at 2009 in WHATWG (Web Hypertext Application Technology Working Group) • IETF and W 3 C get involved at 2011 • W 3 C: http: //dev. w 3. org/2011/webrtc/editor/webrtc. html • IETF: http: //trac. tools. ietf. org/wg/rtcweb/

Real Time Communication (RTC) over the Internet • RTP (real time transport protocol) at 1992 • IETF • Its the standard protocol for today’s Vo. IP • Video/audio conferencing around 1999 • IETF SIP (Session Initiation protocol) • RTC over the web browser • video/audio using plug-in • Gmail Googletalk at 2006 • Web. RTC requires no plug-in and provides fast, low-delay multimedia communication using web browser

Web. RTC • Web Real-Time Communications (RTC) • Add new features to web browser • Using Web as a application development framwework/platform/OS • javascript • HTML 5 • Video/audio • websocket • Cloud computing • client side applications interactive with Server resides at the cloud • Its introduced to provide “Skype” services without any plug-in

Supported browsers

RTC Function in the browser • The RTC function interacts with web app using standard APIs • Peer-to-peer connections with another browser • on-the-wire protocol • TCP • UDP • not HTTP • Signaling server • a route search channel between browsers

Why Web. RTC? • No doubt, the web is becoming a application development platform • The need for RTC for video/audio application is increasing • New audio/video codecs are now included in web browser • Web. RTC can establish Peer-to-Peer communication and take care the network issue • NAT • firewall • Web. RTC can co-exist with current Vo. IP and PSTN network • Web. RTC provideo, audio and data communication channel

Web. RTC protocol Stack Video, Audio, Data, HTTP, Web. Socket, ICE, STUN, TURN, SDP SCTP SRTP DTLS UDP IP Stream Control Transmission Protocol Datagram Transport Layer communications protocol

The SCTP protocol • reliable and unreliable • secure transport • packet reassemble • ordered/not ordered datagram packets • flow and congestion control

Web. RTC architecture

The standard Web. RTC Triangle Webserver + Signaling Server Web. Socket HTTP

What is signaling? • its like when you try to call someone and you need to send out a signal and try to make a connection • exchange peers information • browser version • video/audio codec • its the control band for webrtc peer communication, not the actual video/audio/data transfer channel

Signaling Server Support • No standards are defined from the web. RTC APIs • The signaling can run over HTTP or websocket • ex: using Node. js ws module • Fire. Base • Socket. io

Web. RTC Session Establishment Signaling + web Server 1 3 4 2 6 5 7 8 9 Browser-1 10 Broswer-2 11

How to use Web. RTC? • get. User. Media() • RTCPeer. Connection() • addstream() • Signal offer create/accept Get Local Media Setup Peer connection Attach Media/datachannl to connection Call the peers

Web. RTC APIs • RTCPeer. Connection: • var connections = new RTCPeer. Connection(); • Addstream to your connection • get. User. Media • addstream • connections. onaddstream = function {} //add stream event handler • RTCDatachannel • var datachannel = connections. create. Data. Channel(); • datachannel. send()

get. User. Media() <!DOCTYPE html> <body> <video autoplay> </video> <script > var video //FInd out which browser you are using and get video and audio media from your computer navigator. get. User. Media = navigator. get. User. Media || navigator. webkit. Get. User. Media || navigator. moz. Get. User. Media; //enabling video and audio channels navigator. get. User. Media({ video: true, audio: true }, function (stream) { video = document. query. Selector('video'); //inserting our stream to the video tag video. src = window. URL. create. Object. URL(stream); }, function (err) {}); </script> </body> </html>

RTCPeer. Connection APIs • RTCPeer. Connection() – Returns a new RTCPeer. Connection object. • RTCPeer. Connection. add. Stream() • RTCPeer. Connection. close() • RTCPeer. Connection. create. Data. Channel() – Creates a new RTCData. Channel. • NOW YOU Need a Signaling Server in order to send signaling message to another peer (browser)

Steps to establish Peer connection and add streams on that connection Get Local Media Signaling server Get Local Media Setup Peer connection Attach Media/datachannl to connection Call the peers

Example 1 • Server: Node. js – ws module • run: node server. js • Client: Video/audio • start a simple server to server your HTML file or your media wont be accessible • ex: python –m Simple. HTTPServer

Session description Protocol (SDP) • Got message {"type": "offer", "offer": {"type": "offer", "sdp": "v=0rno=- 8175972782099455380 2 IN IP 4 127. 0. 0. 1rns=-rnt=0 0rna=group: BUNDLE audio video datarna=msid -semantic: WMS 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 grnm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126rnc=IN IP 4 0. 0rna=rtcp: 9 IN IP 4 0. 0rna=ice-ufrag: iyk/KCMo. Kxc. KED 4 arna=ice-pwd: QEh. ADJARm. Luf. NI 78 J 3 LNq 1 q. Urna=fingerprint: sha-256 46: A 3: 6 E: F 1: CA: AE: EE: 32: 77: 8 E: 1 B: DA: 5 B: 8 B: B 5: F 5: 21: C 7: EC: 41: 94: 5 D: 83: 15: BF: 10: D 7: 1 E: 54: 31: 44rna=setup: actpassrna=mid: audiorna=extmap: 1 urn: ietf: params: rtphdrext: ssrc-audio-levelrna=extmap: 3 http: //www. webrtc. org/experiments/rtp-hdrext/abs-send-timerna=sendrecvrna=rtcp-muxrna=rtpmap: 111 opus/48000/2rna=rtcpfb: 111 transport-ccrna=fmtp: 111 minptime=10; useinbandfec=1rna=rtpmap: 103 ISAC/16000rna=rtpmap: 104 ISAC/32000rna=rtpmap: 9 G 722/8000rna=rtpmap: 0 PCMU/8000rna=rtpmap: 8 PCMA/8000rna=rtpmap: 106 CN/32000rna=rtpmap: 105 CN/16000rna=rtpmap: 13 CN/8000rna=rtpmap: 126 telephoneevent/8000rna=maxptime: 60rna=ssrc: 1075132903 cname: ue. TFCy. DAVb 0 Wj. Wmmrna=ssrc: 1075132903 msid: 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 g 9 a 3 b 4 e 61 -e 2 b 842 d 0 -a 6 fd-6 dc 275 a 90 c 81rna=ssrc: 1075132903 mslabel: 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 grna=ssrc: 1075132903 label: 9 a 3 b 4 e 61 -e 2 b 8 -42 d 0 -a 6 fd 6 dc 275 a 90 c 81rnm=video 9 UDP/TLS/RTP/SAVPF 100 101 116 117 96 97 98rnc=IN IP 4 0. 0rna=rtcp: 9 IN IP 4 0. 0rna=ice-ufrag: iyk/KCMo. Kxc. KED 4 arna=icepwd: QEh. ADJARm. Luf. NI 78 J 3 LNq 1 q. Urna=fingerprint: sha-256 46: A 3: 6 E: F 1: CA: AE: EE: 32: 77: 8 E: 1 B: DA: 5 B: 8 B: B 5: F 5: 21: C 7: EC: 41: 94: 5 D: 83: 15: BF: 10: D 7: 1 E: 54: 31: 44rna=setup: actpassrna=mid: videorna=extmap: 2 urn: ietf: params: rtphdrext: toffsetrna=extmap: 3 http: //www. webrtc. org/experiments/rtp-hdrext/abs-send-timerna=extmap: 4 urn: 3 gpp: video-orientationrna=sendrecvrna=rtcpmuxrna=rtpmap: 100 VP 8/90000rna=rtcp-fb: 100 ccm firrna=rtcp-fb: 100 nack plirna=rtcp-fb: 100 goog-rembrna=rtcp-fb: 100 transportccrna=rtpmap: 101 VP 9/90000rna=rtcp-fb: 101 ccm firrna=rtcp-fb: 101 nack plirna=rtcp-fb: 101 goog-rembrna=rtcp-fb: 101 transportccrna=rtpmap: 116 red/90000rna=rtpmap: 117 ulpfec/90000rna=rtpmap: 96 rtx/90000rna=fmtp: 96 apt=100rna=rtpmap: 97 rtx/90000rna=fmtp: 97 apt=101rna=rtpmap: 98 rtx/90000rna=fmtp: 98 apt=116rna=ssrc-group: FID 3022760465 3166375697rna=ssrc: 3022760465 cname: ue. TFCy. DAVb 0 Wj. Wmmrna=ssrc: 3022760465 msid: 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 g b 44 a 5149 -1793 -463 f-a 056 -79 a 1 f 01 ce 26 brna=ssrc: 3022760465 mslabel: 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 grna=ssrc: 3022760465 label: b 44 a 5149 -1793 -463 f-a 056 -79 a 1 f 01 ce 26 brna=ssrc: 3166375697 cname: ue. TFCy. DAVb 0 Wj. Wmmrna=ssrc: 3166375697 msid: 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 g b 44 a 5149 -1793 -463 f-a 056 -79 a 1 f 01 ce 26 brna=ssrc: 3166375697 mslabel: 1 k. Q 3 v. QSp. UGBL 3 y. P 0 sk 9 AFL 40 livs. Uxk. WAW 5 grna=ssrc: 3166375697 label: b 44 a 5149 -1793 -463 f-a 056 -79 a 1 f 01 ce 26 brnm=application 9 UDP/TLS/RTP/SAVPF 127rnc=IN IP 4 0. 0rnb=AS: 30rna=rtcp: 9 IN IP 4 0. 0rna=ice-ufrag: iyk/KCMo. Kxc. KED 4 arna=ice-pwd: QEh. ADJARm. Luf. NI 78 J 3 LNq 1 q. Urna=fingerprint: sha-256 46: A 3: 6 E: F 1: CA: AE: EE: 32: 77: 8 E: 1 B: DA: 5 B: 8 B: B 5: F 5: 21: C 7: EC: 41: 94: 5 D: 83: 15: BF: 10: D 7: 1 E: 54: 31: 44rna=setup: actpassrna=mid: datarna=sendrecvrna=rtcpmuxrna=rtpmap: 127 google-data/90000rna=ssrc: 3113773943 cname: cs. Qa. CKz. MM 0 d. Kv. LLDrna=ssrc: 3113773943 msid: my. Data. Channelrna=ssrc: 3113773943 mslabel: my. Data. Channelrna=ssrc: 3113773943 label: my. Data. Channelrn"}, "name": "user 1"}client-3 -1. js: 13 Got message {"type": "candidate", "candidate": {"candidate": "candidate: 4252197341 1 udp 2122260223 10. 205. 0. 82 61008 typ host generation 0 ufrag iyk/KCMo. Kxc. KED 4 a", "sdp. Mid": "audio", "sdp. MLine. Index": 0}}

Useful Links • More on Web. RTC signaling: • http: //www. html 5 rocks. com/en/tutorials/webrtc/infrastructure/ • Demo link: • https: //github. com/webrtc/samples • http: //www. tutorialspoint. com/webrtc/index. htm • Node. js Websocket • https: //www. npmjs. com/package/nodejs-websocket