Embedded Streaming Media with GStreamer and Beagle Board
Embedded Streaming Media with GStreamer and Beagle. Board ESC-228 Presented by Santiago Nunez santiago. nunez (at) ridgerun. com
Agenda Introduction to Beagle. Board multimedia architecture features. GStreamer concepts OMAP GStreamer hands on exercises DMAI and GStreamer Questions
Introduction Primary objective for this class is to introduce audience to GStreamer on OMAP 3 with hands on exercises This doesn’t class won't cover extensively the APIs available and assumes basic knowledge on C programming, Linux and computer graphics.
Beagle. Board and OMAP 3 architecture Provides architecture with several multimedia features: Cortex A 8 with Neon C 64 x DSP Video accelerators How to utilize the hardware features with the software stack?
Beagle. Board and OMAP 3 architecture Provides architecture with several multimedia features: Cortex A 8 with Neon C 64 x DSP Video accelerators How to utilize the hardware features with the software stack?
Beagle. Board and OMAP 3 architecture Provides architecture with several multimedia features: Cortex A 8 with Neon C 64 x DSP Video accelerators How to utilize the hardware features with the software stack?
GStreamer Streaming media framework – audio and video Close to 200 plug-ins available Higher level than just input / filters / output Networking, audio/video mixed streams, auto data handling Various options utilizing hardware accelerators
GStreamer Overview Elements Sources, filters, sinks Bins and Pipelines Containers, pipeline is the overall bin Pads Element source / sink connection points Caps Capabilities organized by stream type with a set of properties
GStreamer Overview Elements Sources, filters, sinks Bins and Pipelines Plugin Collection of elements Containers, pipeline is the overall bin Pads Element source / sink connection points Caps Capabilities organized by stream type with a set of properties
Simple MP 3 Player Create dynamically using gst-launch Source element reads from a file Filter element converts MP 3 to PWM Sink element passes to ALSA output gst-launch filesrc location=a. mp 3 ! mad ! alsasink
Hands On Exercise 0 Start up Windowing environment with two terminals cd gst . s Period character to source the file named 's'
Hands On Exercise 0 Play audio file a 2 Actual command gst-launch filesrc location=bbb. flac ! flacdec !alsasink You can see the contents of the script cat a 2 There are lots of audio scripts to try a 1, a 2, a 3
Simple Audio Player Source Code Create pipeline, source, filter, sink Set element properties Build into pipeline Connect src and sink pads Setup pipeline event handler End of stream Set pipeline state to play Run
Simple PA System Create dynamically using gst-launch Try it: Source element ALSA audio in r No filters Sink element passes to ALSA output gst-launch alsasrc num-buffers=100 ! alsasink
Keeping Plug-ins Organized Each known plug-in is added to registry Most aspects of plug-in are tracked in the registry Registry support run-in pipeline creation and dynamic filter selection Use gst-inspect to list plug-ins gst-inspect | less gst-inspect filesrc
Hands On Exercise 1 Using gst-inspect, list All plug-ins All video plug-ins Element properties for filesrc plug-in
Hands On Exercise 2 GStreamer video pipelines v 1 – script shown below gst-launch videotestsrc ! ffmpegcolorspace ! fbdevsink Other video scripts v 1, v 2, v 3, v 4 Idea is the same source data, filter data, send data to sink
Performance Data Passing Stream held in buffers with data, timestamp, other info When possible, buffer memory allocated by sink pad Use hardware when data copy is necessary
Performance Data Transformation Cortex A 8 compiler optimization NEON Single Instruction Multiple Data C 64 Video accelerator DMA and other data movers
Davinci Multimedia Application Interface
DMAI and GStreamer Davinci Multimedia Application Interface Exposes OMAP/Davinci hardware using high level of abstraction Stream audio / video Graphics display Hardware optimized frame/data copy
OMAP 3 Codec Engine Isolates users for audio/video codecs from those implementing the codecs Codec can run in several places without the calling application being aware Cortex A 8, NEON, C 64, hardware accelerator Uses DSPLink and DSPBios conventions to support DSP based algorithms dynamically
Video Phone Vision – video phone application can create audio and video GStreamer pipelines for capture / rendering DMAI, Codec engine, DSPLink, C 64 algorithms perform heavy lifting OMAP 3 hardware capabilities utilized Video phone application focused on call management, not streaming data manipulation
GStreamer in 3 Layers
- Slides: 24