Gstreamer overview

From OMAPpedia

Revision as of 15:07, 24 August 2012 by Olivier Naudan (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

[edit] What is GStreamer

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple AV playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. The GStreamer core function is to provide a framework for plugins, data flow and media type handling/negotiation. It also provides an API to write applications using the various plugins. GST elements chain via pads (similar to ports in OpenMAX world) and exchange data between source pads of a data generating element and sink pads of the data consumer element. Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface.

In omap, both GST-OpenMax and GST-Ducati provide elements that interface with omap hardware.

The code for GStreamer-omap can be found in GStreamer Omap repository in gitorious. Check the GStreamer build Instructions to build GStreamer from source code.

For general GStreamer development info check GStreamer developers Info and GStreamer official website

[edit] GStreamer Stack

GStreamer provides plugins across the entire filter graph. GStreamer plug-ins could be classified into

TI specifically focuses on Codecs, Sinks and Filters.

[edit] Omap GStreamer modifications

Gstreamer for omap adds strided video to GStreamer, correspondent to the video format "video/x-raw-yuv-strided". GStreamer library is patched to work with strided format and GST-Ducati and GST-Openmax support strided video.

StrideTransform is an element that is able to convert between strided and non-strided video.

[edit] Source, sink and filter elements

Video sinks
PVRVideosink is the recommended sink to display video. This element is contained in gst-plugins-bad and depends on libpvr2d library.
XImagesink can be used as a fallback video sink when libpvr2d is not available.
Video sources
v4l2src can be used as a video source.

[edit] Sample pipelines

[edit] Display webcam video

gst-launch v4l2src device=/dev/video3 ! "video/x-raw-yuv, width=640, height=480,framerate=15/1" ! queue ! ffmpegcolorspace ! pvrvideosink

[edit] GST-Ducati

GST-Ducati is a GStreamer plugin that provides elements to decode and encode video. GST-Ducati uses the hardware accellerated codecs on IVA-HD via the codec-engine API. It uses the libdce (distributed codec-engine) library to use the codec on ducati.

From GLP1.5 on, GST-Ducati encoders and decoders should be used instead of GST-OpenMax ones.

[edit] Resources

GST-Ducati and libdce code can be downloaded from GStreamer-omap repository. Source for Debian packages are also available at Ubuntu-omap repository.

[edit] Encoders and decoders elements

The following elements are provided by GST-Ducati plugin:

[edit] Sample gst-launch pipelines

[edit] Decoding

GST-Ducati decoders are automatically chosen to play video. (Use option -v to see what elements are being used):

gst-launch playbin2 uri=file://home/user/sample.mp4
gst-launch filesrc location=/home/user/sample.mp4 ! decodebin2 ! pvrvideosink
gst-launch filesrc location=/home/user/sample.mp4 ! qtdemux ! h264parse ! ducatih264dec !  pvrvideosink

Note: On Ubuntu 11.04, there are two additional video sinks:

[edit] Encoding

Encode webcam video to file:

gst-launch -e v4l2src ! "video/x-raw-yuv, width=640, height=480,framerate=30/1" ! queue ! ffmpegcolorspace ! ducatih264enc ! h264parse ! qtmux ! filesink location=sample.mp4

[edit] Encode + Decode

The following pipeline shows the encode/decode latency:

gst-launch -v v4l2src ! stridetransform ! tee name=t \
   t. ! queue ! dri2videosink sync=false \
   t. ! queue ! ducatih264enc rate-preset=low-delay profile=100 ! h264parse ! ducatih264dec ! queue ! pvrvideosink sync=false

[edit] Reworking an existing OMX-based pipeline

In the case you have previously working pipelines using GST-OpenMax encoders and decoders, there are some general rules of thumb to make your pipeline work with GST-Ducati and latest GStreamer-Omap:

[edit] More samples wanted

Stream Video from File

Stream Video from Webcam

Play Video Stream

[edit] GST-OpenMax

In GST-OpenMax, GStreamer plugin interfaces with OMX codecs, drivers or transforms based on the functionality. The TI GST- OMX SW stack is as follows.

GSTFlow.jpg

[edit] GST Buffer Flow

This section provides more details on the exact interfaces between a GST plugin (as in GSTOMX_Base_filter) and OpenMAX component at different stages of the life cycle. This description is based on a generic filter behavior (GSTOMX_Base_filter) that is applicable for video and audio decoders/encoders.

GST-BufferExchange.jpg

[edit] Example TI-OpenMAX filter graph for AV Capture

The following diagram describes a full filter graphs and associations between GST, TI SW and HW components taking AV capture as example

GST-AVCapture.jpg

[edit] GST – OMX integration issues and Impact:

[edit] Impact:

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox