Embedded Streaming Media with GStreamer and Beagle Board

Embedded Streaming Media with GStreamer and Beagle. Board ESC-321 Presented by Todd Fischer todd. [email protected] com

Agenda • Introduction to Beagle. Board multimedia architecture features. • GStreamer concepts • OMAP GStreamer hands on exercises • DMAI and GStreamer • Open. MAX and GStreamer • Questions

Introduction • Primary objective for this class is to introduce audience to GStreamer on OMAP 3 with hands on exercises • This doesn’t class won't cover extensively the APIs available and assumes basic knowledge on C programming, Linux and computer graphics.

Beagle. Board and OMAP 3 architecture • Provides architecture with several multimedia features: • Cortex A 8 with Neon • C 64 x DSP • Video accelerators • How to utilize the hardware features with the software stack?

GStreamer • Streaming media framework – audio and video • Close to 200 plug-ins available • Higher level than just input / filters / output Networking, audio/video mixed streams, auto data handling • Various options utilizing hardware accelerators

GStreamer Overview • Elements • Sources, filters, sinks • Bins and Pipelines • Containers, pipeline is the overall bin • Pads • Element source / sink connection points • Capabilities organized by stream type with a set of properties

GStreamer Overview • Elements • Plugin • Collection of • Sources, filters, sinks elements • Bins and Pipelines • Containers, pipeline is the overall bin • Pads • Element source / sink connection points • Capabilities organized by stream type with a set of properties

Simple MP 3 Player • Create dynamically using gst-launch • Source element reads from a file • Filter element converts MP 3 to PWM • Sink element passes to ALSA output gst-launch filesrc location=a. mp 3 ! mad ! alsasink

Hands On Exercise 0 • Start up Windowing environment with two terminals cd gst. src Period character to source the file e

Hands On Exercise 0 • Play audio file a 2 • Actual command gst-launch filesrc location=bbb. flac ! flacdec ! alsasink

Simple Audio Player Source Code • Create pipeline, source, filter, sink • Set element properties • Build into pipeline • Connect src and sink pads • Setup pipeline event handler • End of stream • Set pipeline state to play • Run

Simple PA System • Create dynamically using gst-launch • Source element ALSS audio in • No filters • Sink element passes to ALSA output gst-launch alsasrc num-buffers=100 ! alsasink

Keeping Plug-ins Organized • Each known plug-in is added to registry • Most aspects of plug-in are tracked in the registry • Registry support run-in pipeline creation and dynamic filter selection • Use gst-inspect to list plug-ins gst-inspect | less gst-inspect filesrc

Hands On Exercise 1 • Using gst-inspect, list • All plug-ins • All video plug-ins • Element properties for filesrc plug-in

Hands On Exercise 2 • GStreamer video pipelines gst-launch videotestsrc ! ffmpegcolorspace ! fbdevsink

Performance Data Passing • Stream held in buffers with data, timestamp, other info • When possible, buffer memory allocated by sink pad • Use hardware when data copy is necessary

Performance Data Transformation • Cortex A 8 compiler optimization • NEON • Single Instruction Multiple Data • C 64 • Video accelerator • DMA and other data movers

Davinci Multimedia Application Interface

DMAI and GStreamer • Davinci Multimedia Application Interface • Exposes OMAP/Davinci hardware using high level of abstraction • Stream audio / video • Graphics display • Hardware optimized frame/data copy

OMAP 3 Codec Engine • Isolates users for audio/video codecs from those implementing the codecs • Codec can run in several places without calling application being aware • Cortex A 8, NEON, C 64, hardware accelerator • Uses DSPLink and DSPBios conventions to support DSP based algorithms dynamically

Video Phone • Vision – video phone application can create audio and video GStreamer pipelines for capture / rendering • DMAI, Codec engine, DSPLink, C 64 algorithms perform heavy lifting • OMAP 3 hardware capabilities utilized • Video phone application focused on call management, not streaming data manipulation

Open. MAX and GStreamer • Addresses codecs using hardware accelerators • GStreamer integration via gst -openmax • All Open. Max codecs available as GStreamer plug-ins • Initial focus on high performance codecs

Open. MAX and OMAP 3 • ARM Inc has Cortex-A 8 / NEON DL implementation • Texas Instruments has AAC and MPEG 4 IL / DL support

GStreamer in 3 Layers