AudioVideo Interconnectivity domain and analysis of interoperability standards

Audio/Video Interconnectivity domain and analysis of interoperability standards BY Khaleel Ali COMP 529

Overview of the presentation Different worlds of electronics l Brief history of PC including Statistics l UPNP l l Description, Implementation and problems l Digital Living Network Alliance (DLNA) l Some UPn. P products l HAVI l Description, l Conclusion Implementation and problems

Three different worlds l Personal computer or PC l Monitor, l Consumer electronic products or CE l Stereo, l printer, scanner, etc receiver, DVD player etc And Mobile l Laptop, PDA, cell phone etc

Examples of some Consumer electronic products (CE)

Examples of some Consumer electronic products (CE)

Mobile world

WHY IS THIS HAPPENING NOW AND WHAT ARE FACTORS THAT ARE CAUSING THIS INTERCONNECTIVITY OF PC, CE AND MOBILE DEVICES ?

First factor l Price of PC has dropped to the lowest level in history. l 10 years ago, cost of computers was around $3000 or more. Now one can buy a good system for about $300. l e. Machine Cel-2. 6 GHz Desktop + 17" CRT + Printer $210 after $400 MIR from Bestbuy. com l Bundling l Extra PCs with online access services rebates for signing up for an online service

Results of first factor By the end of 1999, 52% of U. S households had at least one PC, and 25% had multiple PCs according to International Data Corporation. l By 2004 IDC expects 62% of households in U. S. will have at least one PC and 43% will have multiple PCs. l Most households have at least one CE product. l

Second factor l Increasing growth of home-based offices l More and more people are creating home offices for their own businesses and for completing tasks for their fulltime jobs. l This subsegment of the home networking marketplace is less price sensitive and much more focused on the productivity returns of a given networking peripheral.

Third factor l Increasingly Knowledgeable Consumers driving new applications. l l Consumers are more educated now than ever about the potential that PCs represent. No longer satisfied with just word-processing and e-mail, many consumers now demand Internet sharing, file sharing, gaming both within the same household and across the Internet, peripheral sharing including networked printers and even DVD video applications. Consumers using the fullest extent of the Internet are early adopters, their needs and focus on a wide variety of applications is already having an impact on how standards are defined and devices are designed and manufactured. Consumers are acquiring, viewing, and managing an increasing amount of digital media on devices in the CE, mobile and PC domains. They want to enjoy this content easily and conveniently – regardless of the source – across different devices and locations in the home.

Results of second and third factors l Users want faster home networks and access to internet

Now can the three worlds ever merge? l Personal computers (PC) and consumer electronic products (CE) have always been viewed as separate entities. l l l But technologies exist that can interconnect PC with CE. l l Speakers for computers, and stereo in a single room. TV and then a monitor connected to a PC in a single room. Common Digital media formats EX MP 3, WAV, DVD, MPEG etc We can transfer media using: l l Wired and wireless networks as a backplane for device connectivity. Broadband Internet connections for high-speed access to Internet-based content.

So what was causing the delay in convergence? l Standards were missing l l Consumers want l l l a widely-adopted, standards-based network technology to allow devices from the PC and CE worlds to offer their functionalities and advertise their services to other network devices. Products designed for the home should be easy to install, provide obvious user value and be affordable. Digital home products must interoperate with each other and with existing consumer electronic devices such as TVs and stereos. Product Developer’s Dilemma l l Open industry standards are often too flexible – products built by different vendors all too often fail to interoperate well. Design choices should be narrowed through industry consensus to better achieve interoperability. Current end-to-end solutions based on proprietary vertical implementations bring products to market early but have little impact on rapidly establishing a new category of products.

THE STANDARDS HAVE ARRIVED

First standard UPn. P l Universal Plug and Play Device Architecture (UPn. P) Introduced in 1999 and backed by Microsoft, UPn. P technology has been the preferred device discovery and control protocol. l UPn. P is more than just a simple extension of the plug and play peripheral model. There are more than 738 vendors in consumer electronics, computing, home automation, home security, appliances, printing, photography, computer networking, and mobile products. l Examples of UPn. P devices l Scanners, cameras, CD players, internet gateways etc l Digital Media Adapters (DMAs), which allow the user to view content from the PC on the TV. l SOON Garage door openers, lights, and switches will be available with UPn. P technology. l

Network of devices UPn. P

So what is UPn. P? l UPn. P l l Supports peer-to-peer architecture that allows network connectivity of intelligent appliances, wireless devices, and PCs of all form factors. Also supports DHCP and DNS servers and are used only if available on the network. (optional) offers Easy-to-use, flexible and standard-based connectivity to ad-hoc or unmanaged networks whether in home, small business, public spaces, or attached to the Internet. Is designed to support zero-configuration, "invisible" networking, and automatic discovery device categories from a wide range of vendors. l l A device can dynamically join a network, obtain an IP address, convey its capabilities, and learn about the presence and capabilities of other devices. A device can leave a network smoothly and automatically without leaving any unwanted state behind.

What is UPNP? (cont. . ) l UPn. P l leverages TCP/IP, UDP, HTTP, SOAP, the Document Object Model, and XML and the Web technologies to enable networking in addition to control and data transfer. l uses IP internetworking because l of its proven ability to span different physical media l to enable real world multiple-vendor interoperation l and to connect with the Internet and many home and office intranets. l Further, via bridging, UPn. P accommodates media running non-IP protocols when cost, technology, or legacy prevents the media or devices attached to it from running IP.

The UPn. P protocol stack

Devices in UPn. P l UPn. P devices l are a logical container with a unique type. l provide self-describing information such as the manufacturer, model name, and serial number. l offer any number of services, each service with its own unique service type. l services provide the real functionality. l may also contain other devices l logical composition of devices allows the embedded device to be discovered and used independently from the main container device.

Class diagram of the UPn. P Device

Device and their services in UPn. P l l UPn. P device is similar to an object, the services are like interfaces supported by the object, and the actions in a service are like functions in an interface. Each service in a UPn. P device may contain l any number of actions that Can have a set of input arguments l return value l l l Has a unique Service ID Uniform Resource Identifier (URI) variables that represent that service's current status

Class diagram of UPn. P Service

Control Points in UPn. P l l l A control point is a network entity that invokes a device's functionality. In client/server computing terms, the control point is the client and the device is the server. Control points invoke actions on services, providing any required input parameters and receiving any output parameters and possibly a return value. A control point discovers devices, invokes actions on a device's services, and subscribes to event notifications. A device, on the other hand, responds to invoked actions, sends events when state variables change, and supports a Web page for administrative control

A control point invoking an action on a service

How does UPn. P work? 1. Addressing l 2. Discovery l 3. Control points can now send action messages to device(s) Eventing l 6. Device sends its information to control point Control l 5. Control points and/or devices must discover each other. Description l 4. Device requests IP from DHCP if available. Changes in device status is sent to control point Presentation l Device settings

Addressing l l l is the process by which a device automatically acquires an IP address. is the first step in the operation of a UPn. P device. allows a device to join the network and to prepare to communicate with other UPn. P devices and control points. protocols are built in to UPn. P devices which allow devices to join an IP network dynamically and to acquire an address without user configuration. takes into account whether a device is operating in an unmanaged or managed network. l A device first attempts to contact a DHCP service to acquire an address. If it fails to locate a DHCP server, the device uses Auto. IP, which enables devices to select addresses without a DHCP server being present to assign that address. The addresses are taken from a set of well-known, non-routable addresses. DHCP is a UDP-based protocol, while Auto-IP uses the Address Resolution Protocol (ARP).

Description l l l Description allows devices to list the functionality they provide. Description of devices and their services are contained in XML-based description documents. Device description document contains device information such as manufacturer, make, model, and serial number; a list of services provided by the device; and a list of embedded devices. A service description document contains detailed information about a device's service, the actions that service provides, and the service's parameters and return value. The description documents are used by control points in the device discovery process to learn more about a device.

Description diagram

Discovery l defines how a device announces its presence, and how control points discover it. l l l A UPn. P device is like a mini network server that can be discovered and controlled by a UPn. P control point. process enables control points to find devices and services of interest and retrieve information about them. Is done by Simple Service Discovery Protocol (SSDP). l l l SSDP extends the HTTP (Hypertext Transfer Protocol) header to provide a simple multicast-based discovery protocol. Once a device has acquired an IP address, that device periodically advertises itself and its services on the network. A device includes a URL of its device description XML document in its advertisement and discovery responses. That URL provides control points with the information those control points need to retrieve the device and service descriptions. The description documents are retrieved by control points and parsed to understand all about that device. A vendor can add extensions beyond the basic functions and include those extensions in the description docs. That extension mechanism allows a control point to prefer an optional or vendor-specific interface over the standard one.

Discovery (Cont. . ) l Every service contained in a device maintains three URLs that provide the information necessary for control points to communicate with that service: l l l The Control. URL is where control points post requests to control the service. A UPn. P device vendor specifies one for each device. The Event. Sub. URL is where control points post requests to subscribe to events. There is one Event. Sub. URL for each service in a device. The Description. URL tells control points the location to retrieve the service description document from.

Discovery diagram

Control l Control is the UPn. P phase when control points invoke the actions of a device's services. l l l When a service receives a control message, that service acts upon that message. UPn. P relies on the Simple Object Access Protocol (SOAP) for device control. SOAP brings together XML and HTTP to provide a Web-based messaging and remote procedure call mechanism. XML expresses the message content, while HTTP sends messages to their destination. SOAP is specified as a set of conventions that govern the format and processing rules of SOAP messages.

SOAP l l The SOAP message is the basic unit of communication between peers. SOAP messages are written in XML, making SOAP platform independent: any system capable of creating and parsing XML documents can send and receive SOAP messages. The power of XML allows SOAP messages to be fairly complex in structure and to transmit highly complex data types. SOAP consists of four parts: l l The SOAP envelope: An XML schema that defines a framework for describing what is in a message, how to process it, and whether that processing is optional or mandatory. The SOAP encoding rules Another XML schema that defines a set of rules for expressing instances of application-defined data types. The SOAP binding: A convention for using different transport protocols. SOAP can potentially be used in combination with a variety of other transport protocols (however SOAP messages are most commonly carried by HTTP). The SOAP RPC representation: A convention for representing remote procedure calls and responses.

Eventing l l An event is a message from a service to subscribed control points. Events keep control points informed of changes in state associated with a service. Control points subscribe to events, and the service notifies interested control points of events. A control point interested in receiving notification of state variable changes subscribes to an event source by sending a subscription request that includes: l l l The service of interest A URL to which the events can be sent A subscription time for the event notification A subscription is a request to receive all state variable changes. If a service accepts the subscription request, that service responds with a unique subscription identifier and the duration for the subscription. The duration specifies the length of time that the subscription is valid and maintained by the service.

Eventing (cont. . ) l l l All subscribers are sent all event messages. Subscribers receive event messages for all evented variables, and event messages are sent regardless of the reason the state variable changed. UPn. P's only guarantees that a message gets delivered to a subscriber is that it the eventing protocol (GENA) uses TCP for transport. When a subscription expires, the subscription identifier becomes invalid, and the publisher stops sending event messages to that subscriber. All subscriptions must be renewed periodically for the control points to continue to receive notifications. To keep a subscription active, a control point must send a renewal message before that subscription expires. When a control point no longer wants to receive events from a service, the control point cancel its subscription.

Presentation l l l Presentation is the process where a device presents a browser-based user interface for manual user control and to allow viewing of that device's status. Each UPn. P device contains an internal HTTP Web server and may provide a Web page for browser-based clients. That Web page serves as the device's manual interface that complements the device's programmatic SOAPbased control interface. The browser-based interface is used to control the device, to change operational parameters, to view device and service information, or for any other device-specific functionality implemented by the manufacturer. The presentation page provides a simple, consistent way to work with UPn. P devices and requires no custom software.

DIGITAL LIVING NETWORK ALLIANCE (DLNA) AND ITS VISION • To have a computer (PC) or consumer electronic product (CE) that controls the whole house entertainment system(s). • Reasons: – Convenience and availability • All movies, music, pictures etc available on any device in network. • Access to all equipment that are part of the network. • Management of all equipment that are part of the network. • Remote sharing of resources.

DLNA expectations l l The Internet, mobile and broadcast islands will integrate through a seamless, interoperable network that will provide a unique opportunity for manufacturers and consumers alike. Digital homes will contain one or more intelligent platforms, such as an advanced set-top box (STB) or a PC. These intelligent platforms will manage and distribute rich digital content to devices such as TVs and wireless monitors from devices such as digital stills cameras, camcorders and multimedia mobile phones.

DLNA interoperability guidelines V 1. 0

DLNA interoperability guidelines V 1. 0 (cont. . )

DLNA interoperability guidelines V 1. 0 (cont. . )

UPn. P products in market

Problems with UPn. P

Problems with UPn. P (cont. . ) • DVD resolution is 720 x 540. • The gross transfer rate is generally somewhere between 12 and 14 Mbit/s, making the real rate around 5. 5 to 6 Mbit/s. That's enough to play MPEG-1/2 videos in (S)VCD and mid-range DVD quality. This test was conducted by Tomshardware. com • Price of UPn. P entertainment center and other products is high.

SECOND STANDARD HAVI l Home Audio Video interoperability (HAVi) l l provides a home networking standard for seamless interoperability between digital audio and video consumer devices. The main reason for having a dedicated HAVi is the exchange of high quality digital video and audio signals. HAVi is an initiative from eight major Consumer Electronics companies: Grundig AG, Hitachi, Matsushita , Panasonic, Philips , Sharp, Sony Corp, Thomson Multimedia and Toshiba Corp. More than 30 companies are involved in the development of HAVI.

HAVi architecture l l The HAVi architecture is open, scaleable in implementation complexity, platform-independent and language neutral, i. e. HAVi can be implemented in any programming language and on any CPU or real-time operating system. In order to be able to handle both commands and multiple digital audio and video streams, HAVi uses the digital IEEE-1394, 1394 a and future extensions, and IEC 61883 interface standards network. l l IEEE-1394 currently provides a bandwidth of up to 800 Mb/s and is capable of isochronous communication which makes it suitable to simultaneously handle multiple real-time digital AV streams. Longer transmission distances under the IEEE 1394 standard are near to completion and will allow the IEEE-1394 network to span multiple rooms in a home.

A HAVi network

Promises of HAVi l Time synchronization between different devices. l l TV set might get the correct time from the broadcast stream and the other devices can query the TV and set their own clocks according to it. Recording can be as simple as just browsing the program information, selecting the desired program and pressing one button to activate recording. automatic directing of an oncoming videophone call to the TV screen or part of it and muting all other sounds. camera placed outside the door detects movement, the picture is automatically connected to the TV screen notifying the user about a possible visitor.

Promises of HAVi (cont. . ) l l The interoperability of HAVi devices not complex. Devices are hot-pluggable and they automatically announce their presence and capabilities to other devices and configure themselves when connected to the Network. l l l This saves the user from reading installation instructions and configuring network addresses and drivers. HAVi standard promises to be future proof by maintaining current functionality while making it easy to upgrade and add new capabilities. Non-HAVi devices can also be connected to the network if at least one of the HAVi devices supports the interface the legacy device provides. Two categories of legacy devices are l l non-1394 devices not supporting the HAVi Architecture

HAVi’s Future-Proof Support l the HAVi Architecture supports future devices and protocols through several software-based mechanisms. These include: l l Each HAVi-compliant device may contain persistent data concerning its user interface and device control capabilities. l l persistent device-resident information describing capabilities of devices a write-once, run-everywhere language (Java), used for software extensions a device independent representation of user interface elements This information can include Java bytecode that can be uploaded and executed by other devices on the home network. As manufacturers introduce new models with new features they can modify the bytecode shipped with the device. The new functionality added to the bytecode mirrors the new features provided by the device. Similarly new user interface elements can be added to the stored UI representation on the device.

Plug-and-Play Support l l l Home network consumer devices are easy to install, just connect the cables and no configuration is necessary. In the HAVi Architecture, a device configures itself, and integrates itself into the home network, without user intervention. Home networking technology offers “hot” plug-and-play (not requiring the user to switch off devices), and safe and reliable connections. Low-level communication services provide notification when a new device is identified on the network. Installing a device on the home network will be simpler than stand-alone installation since new devices can obtain configuration information from those already on the network. l a DTV receiving time signals via digital broadcast.

HAVi Architecture l The HAVi architecture can be divided into several different layers. l On the bottom l l l Medium layer l l l there is always vendor specific hardware and software Application Programming Interface (API) that HAVi is built upon. there is the connecting IEEE 1394 wiring, which HAVi devices use as a connecting medium. a media manager for IEEE 1394 is needed as well as a messaging system. On top of the messaging system there are several software modules: Registry, Event Manager, Stream Manager, Resource Manager, Device Control Modules (DCM) and DCM managers. Top layer l Havlets and application for user interaction with UI mechanism

Diagram of HAVi Architecture

HAVi Architecture l l l l 1394 Communication Media Manager – allows other software elements to perform asynchronous and isochronous communication over 1394. Messaging System – responsible for passing messages between software elements. Registry – serves as a directory service, allows any object to locate another object on the home network. Event Manager – serves as an event delivery service. An event is the change in state of an object or of the home network. Stream Manager – responsible for managing real-time transfer of AV and other media between functional components. Resource Manager – facilitates sharing of resources and scheduling of actions. Device Control Module (DCM) – a software element used to control a device. Within a DCM code unit are code for the DCM itself plus code for Functional Component Modules (FCMs) for each functional component within the device. DCM Manager – responsible for installing and removing DCM code units on FAV and IAV devices.

DCM is the heart of HAVi l The first DCM characteristic is how the DCM is obtained by the controller: l l l The second characteristic is whether a DCM is platform (controller) dependent or platform independent: l l l embedded DCM – a DCM that is part of the resident software on a controller. uploaded DCM – a DCM that is obtained from some source external to the controller and dynamically added to the software on the controller. native DCM – a DCM that is implemented for a specific platform, it may include machine code for a specific processor or access platform specific APIs. bytecode DCM – a DCM that is implemented in Java bytecode. Finally, DCMs can be distinguished by their functionality (or, conversely, their range of use): l l standard DCM – a DCM that provides the standard HAVi APIs. Such a DCM provides basic functionality but is able to control a wide range of devices. proprietary DCM – a DCM that provides vendor-specific APIs (in addition to the standard HAVi APIs). Such a DCM would offer additional features and capabilities over a standard DCM but could control a narrower range of devices, perhaps only a specific device or model.

HAVi devices

HAVi devices l HAVi devices are classified into four categories l l l Full AV devices (FAV), Intermediate AV devices (IAV) Base AV devices (BAV) and Legacy AV devices (LAV). HAVi compliant devices fall into the first 3 categories and all other devices belong to the 4 th category. FAVs and IAVs are controlling devices in the HAVi network. l BAVs and LAVs are the devices they control.

Full AV devices (FAV), l l l A Full AV device contains a complete set of the software elements comprising the HAVi Architecture. FAV supports a complex software environment. FAV devices have a runtime environment for Java bytecode. l l This allows an FAV to upload bytecode from other devices and so provide enhanced capabilities for their control. Example of FAV devices are l l Set Top Boxes (STB), Digital TV receivers (DTV), general purpose home control devices and Home PC’s.

Intermediate AV (IAV) l Intermediate AV devices are lower in cost than FAV devices and more limited in resources. l They do not provide a runtime environment for Java bytecode. l Cannot act as controllers for arbitrary devices within the home network. l IAV may provide native support for control of particular devices on the home network.

Base AV devices (BAV) l l These devices implement future-proof behavior by providing uploadable Java bytecode, but do not host any of the software elements of the HAVi Architecture. These devices can be controlled by an FAV device via the uploadable bytecode or from an IAV device via native code. The protocol between the BAV and its controller may or may not be proprietary. Communication between a FAV or IAV device and a BAV device requires that HAVi commands be translated to and from the command protocol used by the BAV device.

Legacy AV devices (LAV) l l LAV devices are devices that are not aware of the HAVi Architecture. These devices use proprietary protocols for their control, and quite frequently have simple controlonly protocols. Such devices can work in the home network but require that FAV or IAV devices act as a gateway. Communication between a FAV or IAV device and legacy device requires that HAVi commands be translated to and from the legacy command protocol.

HAVI configuration

IAV or FAV as controller

IAV or FAV as Display

Interoperability in the HAVi Architecture l The first and foremost goal of the HAVi Architecture is to support interoperability between AV equipment. l l Because of the need to support existing devices, and because of product cost considerations, the HAVi Architecture supports two levels of interoperability. l l l This includes existing equipment and future equipment. Level 1 and Level 2. The flexibility of choosing different levels of interoperability gives vendors the freedom to design and build devices at all points on the cost/capability spectrum.

Level 1 Interoperability Level 1 interoperability addresses the general need to allow existing devices to communicate. l Level 1 interoperability defines and uses: l la generic set of control messages (commands) that enable one device to talk to another device l and a set of event messages that it should reasonably expect from the device.

Level 1 Interoperability (cont. . ) l l l Device discovery l HAVi Architecture has adopted is to utilize Self Describing Device (SDD) data, required on all FAV, IAV and BAV devices. l SDD data contains information about the device which can be accessed by other devices. Communication l a general communication facility allowing applications to issue requests to devices. l This service is provided by the HAVi Messaging Systems and DCMs. l The application sends HAVi messages to DCMs, the DCM then engages in proprietary communication with the device. HAVi message set: l is a well defined set of messages that must be supported by all devices of a particular class. l This ensures that a device can work with existing as well as future devices, irrespective of the manufacturer. l The HAVi message set includes those messages used for the DDI protocol and so allows DCMs (and applications) to construct a UI on display-capable IAVs and FAVs.

Level 2 Interoperability l l l l l To support non-standard features of existing products and to support future products, the HAVi Architecture allows uploaded DCMs as an alternative to embedded DCMs. Uploaded DCMs are implemented in Java bytecode. Level 2 only requires that one device provide a runtime environment for the uploaded DCM obtained from the new device. The concept of uploading and executing bytecode also provides the possibility for applications called havlets. Havlets may be device specific Havlets can be uploaded and installed by each FAV device on the network. Havlet can be supplied by DCMs and/or the user via Level 2 UI. Display-capable FAVs will allow a user to upload and execute the havlet of any DCM or Application Module.

Security in HAVi uses a two-level protection scheme. l When a software element is created it is assigned an access level which is one of trusted or untrusted. l When one software element sends a request to another software element the receiver decides whether or not to honor the request by examining the access level of the requester. l

Security in HAVi l l All uploadable DCM code units shall be signed. The HAVi Certification Authority (HCA) also issues several pairs of private key and public key to each vendor on request, with the HCA's digital signature for the vendor unique public key. A HAVi-signed code unit includes some specific files necessary for authentication in the JAR file. When an FAV is to install a HAVi-signed code unit, it has to verify the code unit with these files in advance to install the software element.

Certificate File The signature file is verified with the ‘vendor-unique public key’ which corresponds to the vendor-unique private key that is used to generate the signature. l The ‘vendor-unique public key’ must be certified by the HCA. l The verifier module in an FAV can find the ‘vendor-unique public key’ to verify the signature in this file, and shall verify whether the public key is trusted. l

PROBLEMS WITH HAVI l l HAVi as a technology hasn’t been widely tested and utilized in real environments. One of the most important goals is platformindependent interoperability. l Fire. Wire still isn’t as flexible and has proven to be very complicated to implement. l l l The only guarantee is that devices of the same brand same kind of proprietary programming will most likely work together. Until all the major problems with Fire. Wire have been solved, they will hinder HAVi. One important issue when dealing with home entertainment is digital rights management. l HAVi has no digital rights management

PROBLEMS WITH HAVI (cont. . ) l l l Faulty device might get an update containing a virus which it then uploads to every other device. Cost of HAVi products are high. Very few products on market as of yet. Linux and java have problems such as real-time resource management and making the memory footprint small enough. Competition by UPn. P, and now UPn. P with Pre N technology. Communication medium needed to connect to internet. HAVi doesn’t support XML and SOAP.

CONCLUSION HAVi or UPn. P No devices available for HAVi l More problems associated with HAVi l UPn. P is fairly new but functional l UPn. P is in sync with today’s technology l l XML, l l SOAP, WEB, wireless etc Bridge between UPn. P and HAVi is missing or just on paper. Digital management and security in HAVi

REFERENCES l DLNA l l UPn. P Architecture l l l http: //www. dlna. org/news/DLNA_Overview. pdf http: //www. upnp. org/ http: //www. artima. com/spontaneous/upnp_digihome 2. html HAVI l l l http: //www. havi. org/pdf/white. pdf http: //www. tml. hut. fi/Studies/Tik 111. 590/2001 s/papers/jussi_teirikangas. pdf http: //www. xilinx. com/esp/consumer/home_networkin g/pdf_files/havi/complete. pdf