Multimedia Communication Applications What it means What can

Multimedia Communication Applications • What it means ? • What can we do ? • How can we do ? • What we have done ? • Q/A Kundan Singh Internet Real-Time Lab. Columbia University October 4, 2000 © 2000, Columbia University

Multimedia + Communication • • • Audio (voice + music) Video Data • • Internet (IP) Telephone (PSTN) Interactive Vs non-interactive October 4, 2000 © 2000, Columbia University

Applications Yahoo audio chat Internet radio/TV Telephony over Internet Video conferencing Dialpad Instant messaging and presence Unified messaging Email/phone Multi-user games October 4, 2000 © 2000, Columbia University

Approaches …add multimedia to my PSTN, ISDN, ATM, … use it for internet… Telco (ITU-T) …develop and use mbone tools: vic, vat, rat, … Collaborative Work October 4, 2000 …signaling and media are different … …add new services (email, voice mail, fax) accessible from web … IETF Games …add any network support to my multi-user games… Web based services © 2000, Columbia University

Signaling Protocols IETF Telco (ITU-T) H. 320 (ISDN) H. 324 (POTS) H. 323 (LAN Internet) (H. 245, H. 225, H. 450. x, H. 332, …) RTP/RTCP (Media transport) HTTP (Web) RTSP (Streaming) SIP (Session initiation) … RMI October 4, 2000 different approach © 2000, Columbia University

Scenarios • • • Client/server – RTSP media player and RTSP server Distributed infrastructure – Vo. IP Multiparty work - conferencing Messaging - email. . . October 4, 2000 © 2000, Columbia University

Another approach Focus on multimedia communication components instead of specific applications Audio input Different communication protocols (SIP, RTSP, H. 323, …) Storage Audio output Video input October 4, 2000 Display © 2000, Columbia University

What you need ? • Ease of development and experimentation - Examples … write a software which calls me on my phone whenever I get an important mail … … call a number and leave a message. The message is converted to text and sent as email … October 4, 2000 © 2000, Columbia University

What you need ? • • Framework for development Properties: scalable, extensible, interoperable, easy to use Streaming application, Multimedia conferences, Gateways, … Applications API RTP/RTCP, SIP, RTSP, DNS, H. 323, … October 4, 2000 Libraries © 2000, Columbia University

CINEMA Columbia Inter. Net Extensible Multimedia Architecture • • • October 4, 2000 SIP, RTSP (and HTTP) Additional external libraries: RTP, H. 323, … API: C++ and Java © 2000, Columbia University

Authentication Basic and Digest User registration CGI/CPL upload Dynamic session change October 4, 2000 Components User registration SIP/SDP parser Dynamic session change SIP/SDP Parser Ea sy to use ! to be added. . . • Call transfer • Three party call • Instant messaging and presence © 2000, Columbia University

Related work • • API • JTAPI • Java Servlet • JAIN SIP API MASH October 4, 2000 © 2000, Columbia University

CINEMA RTSP media server SIP proxy server SIP/H. 323 gateway rtspd sip 323 LDAP Berkeley DB xml 4 j SIP conferencing sipconf SIP/RTSP unified messaging SIP/MGCP gateway sipum sipgw Open. H 323 CINEMA lib. NT libcine libsip++ Win 32 stub Utilities parsing Basic SIP library SIP UA library PGP PWLib Resparse October 4, 2000 libmixer RTP audio mixer libdict Hash table libdb++ my. SQL intf © 2000, Columbia University

Example: audio conferencing • Types • • October 4, 2000 Centralized Distributed Multicast Local media mixing © 2000, Columbia University

Example: audio conferencing • Types • • Centralized Distributed Multicast Local media mixing A D B October 4, 2000 C © 2000, Columbia University

Example: audio conferencing • Types • • Centralized Distributed Multicast Local media mixing Use multicast for media traffic. October 4, 2000 © 2000, Columbia University

Example: audio conferencing • Types • • Centralized Distributed Multicast Local media mixing A B C Mix audio October 4, 2000 © 2000, Columbia University

Sipconf - 1 SIP based audio conference server • Features • • • October 4, 2000 SIP/SDP and RTP/RTCP Audio mixing – sender does not get his own audio Play-out delay algorithm Web based setup G. 711 A and Mu law, G. 721, DVI ADPCM Multiple simultaneous conferences © 2000, Columbia University

Sipconf - 2 Comparison with other types • • Multicast – deployment Simple endpoints – no local mixing Overall bandwidth Add all and subtract one – O(N); N=participants Extensible – codecs, video Heterogeneous participants Silence suppression - scalability October 4, 2000 © 2000, Columbia University

What else ? SIP based Audio/Video conferencing bridge SIP/PSTN gateway Web based SIP client SIP/RTSP based voice and video mail SIP for instant messaging and presence SIP based email/phone October 4, 2000 © 2000, Columbia University

Missing pieces • • • Qo. S Billing Security Telephone interface. . . October 4, 2000 © 2000, Columbia University

Summary • What it means ? -multimedia + communication • What can we do ? -chat, radio/TV, games, telephony, … • How can we do ? - SIP, RTSP, RTP, H. 323, … • What we have done ? - sipc, sipd, rtspd, libsip++, sip 323, sipum, sipconf, … October 4, 2000 © 2000, Columbia University

Additional material October 4, 2000 © 2000, Columbia University

SIP H. 323 e*phone sipc PSTN sipgw Missing component … gateway SIP/H. 323 SIP audio conference signaling server SIP/RTP based embedded ethernet Desktop SIP/RTSP server based (proxy, based SIPunified redirect, user-agent messaging registrar) SIP/PSTN gateway built-inmixing, audio gatekeeper, play-out fast-start delay, as multiple well asconferences, non. Audio, Authentication, voice messaging, video, white-board, sipanswering cgi, Ldap chat, , machine, unix address and windows book phone device (user-agent) fast-start, web setupmultiple calls, transparent media, web based setup, email and web integration unix and windows October 4, 2000 PBX © 2000, Columbia University

Related work - Parlay • • Technology independent interface for telecom Services developed outside core network Access network function in secure way APIs: IT applications Communication world • Service interface: Network capabilities and information Call management, messaging, user interaction, billing, policy management… • Framework interface: external interface in secure and manageable way • • Service registration, subscription, discovery Authentication, authorization Integrity management and other management support Call model not suited for Internet telephony October 4, 2000 © 2000, Columbia University

Related work - JTAPI • • (Java) Telephony API – connection, call, numbers, … Not suited for Internet Telephony October 4, 2000 © 2000, Columbia University

Related work – SIP-CGI, CPL • • • Control of server behavior SIP-CGI similar to HTTP CGI Call processing language is XML based October 4, 2000 © 2000, Columbia University

Related work – SIP Servlet • • Similar to HTTP (Java) Servlet Advantages: stateful, low overhead, typed API, other Java based services like directory service, database, JMF… Example: location manager, redirection or proxy server, … More close to SIP-CGI October 4, 2000 © 2000, Columbia University

Related work – JAIN • • • Java API for Integrated Networks PSTN, wireless, Internet. Very extensive framework Protocol API spec: ISUP, MGCP, SIP, … Application API spec: JCC/JCAT Call model vs transaction model, JSPA Parlay, SCE/SLEE service creation and logic execution environment, … JAIN SIP API: Stack provider (Jain. Sip. Provider) and Application (Jain. Sip. Listener) October 4, 2000 © 2000, Columbia University

Related work – MASH • • Multimedia architecture for communication and collaboration over scalable and heterogeneous networks using IP multicast Technologies • • SRM Active objects Scalable consensus based bandwidth allocation Media archival and playback Coordination and control Transmission over heterogeneous networks Mash shell OTcl multimedia components Multimedia collaboration tools vic, vat, … October 4, 2000 © 2000, Columbia University

Related work • • • CINEMA concept is close to MASH – SIP and RTSP based API Provides different levels of abstraction • Low level – like JAIN SIP API and SIP Servlet API • High level – Call model for user agents (and servers) Can inter-work with SIP-CGI, CPL October 4, 2000 © 2000, Columbia University

Thank you http: //www. cs. columbia. edu/~kns 10/talks/irt 3. ppt October 4, 2000 © 2000, Columbia University