Overview of videoconference clients in EVO VIC VIdeo

Overview of videoconference clients in EVO

VIC [ VIdeo. Conferencing Tool ] • VIC is default video client of EVO system • based on open source project of Open. Mash consortium • the application is written in C++ and OTcl programming languages • development is focused on following points: - video codecs (H. 261 and H. 263) - performance – IPP (Intel Performance Primitives) Libraries - communication interface for external applications - user’s comfort and GUI

VIC [ H. 261 - encoder ] • new encoder is fully compliant with the H. 261 standard • old encoder in VIC used only INTRA-mode – compression is based on reducing of intra frame spatial redundancy • added was INTER-mode with inter-frame motion estimation and compensation - difference between two subsequent video frames is encoded - compression is based on reducing of inter frame temporal redundancy frame n+1 difference • reduces bitstream (higher compression) - allows to use higher framerate and quality with the same bitstream

VIC [ H. 261 - decoder ] • improvements done on decoder side have influence mainly on compatibility with H. 323 hardware devices • RTP part was changed to avoid artifacts in decoded video from some H. 323 clients and to fix bad positions of macroblocks in video stream from MCUs and Polycom VSX • bug which causes skipping of the last macroblock in RTP packet was fixed (avoids artifacts in decoded video) MCU Codian - before Polycom VSX 7000 - before MCU Codian - now Polycom VSX 7000 - now

VIC [ H. 263 codec ] • compare to H. 261, there are two significant differences: resolution of video signal (standardized formats up to 16 CIF and custom formats up to 2048 x 1152) and level of compression and quality PRESENT STATUS • currently is implemented only INTRA-frame mode • development is focused on compatibility with H. 323 devices, improvement of codec performance, capturing video with larger resolutions • Desktop Sharing Mode - VIC is able to capture PC desktop and send it like H. 263 video • one application as video client and desktop sharing client • advantage of existing infrastructure - signal will pass through network limitations like NAT • higher compression of the signal using H. 263 capabilities

VIC [ H. 263 - Desktop Sharing ] H. 263 XGA (1024 x 768) H. 261 CIF (352 x 288) H. 263 VGA (640 x 480)

VIC [ Users Comfort ] • the same version for Windows, Linux and Mac OS X with the same features • new GUI – more user friendly, intuitive and easier to customize. All technical settings have been removed from the main settings interface window • Speaker Mode - VIC is able to identify and highlight video of current speaker - information about who is the current speaker is generated by reflector and carried by specific RTCP packet (SDES) • new video display modes which automatically arrange video windows on the desktop - all videos in CIF size - all videos in QCIF size - only the video of current speaker in CIF size. - current speaker in CIF size plus all the other videos in QCIF size

VIC [ Users Comfort ] • possibility to save all settings from GUI including position on the desktop • multiple display support for Windows systems - user can choose display, on which the videos from participants will be displayed • Still Image Mode – allows transmitting of static picture instead of video from camera • Minimize Mode – hides small video windows on the main VIC window, saves space on the desktop • and many others improvements like using Fire. Wire cameras on MAC without the IOXperts driver and freezing of video

RAT [ Robust Audio Tool ] • RAT is default audio client of EVO system • based on open source project of University College London • the application is written in C and Tcl/Tk programming languages • new audio codec was added (G. 722) for compatibility with H. 323 devices • priority of development was to unify source code for all supported platforms (Windows, Linux, Macintosh) – required new audio interface for MAC OS X • ongoing development is focused on splitting audio processing and windows management into two separated threads, what increase performance and stability of the application

VIC and RAT in on. EVO Overview of videoconference clients WINDOWS (17 participants)

VIC on LINUX (17 participants) 12

VIC and RAT on MAC OS X (9 participants) On Mac OS X 13

Communication Interface [ e-Learning scenario ] Teacher Student

Communication Interface [ localhost monitoring scenario ] EVO Client RAT VIC ALARM: CPU overloaded Response: decrease framerate, quality, resolution, change video codec, turn off display of big video windows, … Localhost monitoring module LISA

Pocket. VRVS – the mobile solution • Pocket. VRVS is new VRVS videoconferencing client developed for Pocket PC platform • application is written in C++ and Windows CE API • it supports H. 261 video standard and G. 711 (µ-Law) audio standard • user can choose CIF sized still image to be transmitted (JPEG format) • the logic of video encoding was adapted to encoding only still images - the process was optimized to reach the compromise between framerate, CPU usage and fast update of video on remote clients

Pocket. VRVS – the mobile solution • utilization of INTEL Performance Primitives and Intel compiler for optimization of application performance (Discrete Cosine Transform, color model conversions RGB to YUV) H. 323 speaker connection • received and decoded is only video of current – interface this functionality is implemented on reflector’s side • Pocket. VRVS works inside networks with network address translation (NAT/NAPT) • new web based interface for Pocket PC clients (designed for smaller screens) includes full functionality available for desktop clients (meeting scheduler, booking, …) and also possibility to connect H. 323 device • installation package of Pocket. VRVS application with multilanguage OS support is intended directly for Pocket PC platform, desktop computer is not needed

Pocket. VRVS – the mobile solution Control Panel Documentation Talk, Listen, Send and Receive controls Download Pocket. VRVS package Video Screen Remote Video, Own Video in PIP window Booking Session Info Virtual Room, Participant’s name, … Connect to any ongoing meeting Pocket. VRVS web interface Pocket. VRVS application GUI Program Menu Connection and Session Settings

Conclusions Work on new video codecs H. 263, H. 264 in VIC • higher quality and compression of video signals with higher resolution Implement software echo canceller into RAT • improvement of the quality of transmitted audio signals using intelligent algorithm for echo and noise suppression Merging VIC and RAT into one application • efficient processing of incoming video and audio streams by one application (common GUI) + video and audio synchronization Interconnection between VRVS/EVO, RAT and VIC • support of various videoconferencing scenarios (education, tutorials, meetings); reaction on alarms Support of live video transmission from Pocket. VRVS § send real video from cameras for Pocket PC platform Thank you for your attention

Additional Slides

VIC [ H. 261 - encoder ] • Block scheme of H. 261 encoder with highlighted newly developed parts • Diamond searching algorithm Median filter defined in H. 261 standard • Searching area +/- 7 • SAD sum of absolute differences • better compatibility with H. 323 devices – solutions on encoder side: • equal RTP packet size (via MBA stuffing) – avoids packet reordering and “freezing” of decoded video • fixed H. 261 payload RTP headers

VIC [ H. 263 codec ] • compare to H. 261, there are two significant differences: Resolution of video signal • standardized picture formats: sub-QCIF (128 x 96), QCIF (176 x 144), CIF (352 x 288), 4 CIF (704 x 576), 16 CIF (1408 x 1152) • custom picture formats: resolution up to 2048 x 1152 Level of compression and quality • Arithmetic Coding instead of VLC - significantly fewer bits produced • Unrestricted Motion Vector Mode - larger vectors and vectors can point outside the picture • Advanced Prediction Mode - 4 8 x 8 vectors instead of one 16 x 16 vector - results in less blocking artifacts • PB-frames mode - two pictures encoded as one unit • enhanced error resilience capabilities • half pixel precision used for motion compensation + many others

VIC [ The Performance ] • IPP (Intel Performance Primitives) libraries were used for better performance of video codecs in VIC • IPP allows to replace a part of code, which is heavy on computation – DCT, Motion compensation, … • automatic processor type detection • available for Linux and Windows • Intel compiler optimizes compilation for target processor • implementation of IPP functions in some H. 26 x encoder and decoder parts leads to significant decrease in CPU usage

Communication Interface • interface allowing to receive commands from external applications - like VRVS/EVO client - for interaction with VIC and RAT during runtime • communication interface uses existing RTCP channel of VIC and RAT applications – doesn’t need additional channel • remote commands are carried by specific Application Defined (APP) RTCP packets that are understood and subsequently processed only by VIC and RAT applications and ignored by the others • useful in conference scenarios, when privileged user can control audio and video clients of other participants (e-Learning: teacher vs. students) • VRVS/EVO client can take appropriate action as reaction on alarm notifications when monitored parameters (system or network) will go beyond a preset threshold