GStreamer - 2020.2 English

Versal ACAP VMK180 Targeted Reference Design (UG1432)
Document ID
UG1432
Release Date
2021-01-08
Version
2020.2 English

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple playback and audio/video streaming to complex audio mixing and video processing.

GStreamer uses a plug-in architecture that makes the most of GStreamer functionality implemented as shared libraries. The GStreamer base functionality contains functions for registering and loading plug-ins and for providing the fundamentals of all classes in the form of base classes. Plug-in libraries get dynamically loaded to support a wide spectrum of codecs, container formats, and input/output drivers.
Table 1. GStreamer Plug-ins
Plug-in Description
v4l2src v4l2src can be used to capture video from V4L2 devices such as MIPI.

Example pipeline: gst-launch-1.0 v4l2src ! kmssink

This pipeline shows the video captured from a /dev/video0 and rendered on a display unit.
Kmssink The kmssink is a simple video sink that renders raw video frames directly in a plane of a DRM device.
Example pipeline: 
gst-launch-1.0 v4l2src ! “video/x-raw, format=NV12, width=3840, height=2160” !
      kmssink

Video Buffer Management

The application uses the DMABUF framework for sharing buffers between a display device (DRM), and a video capture device (V4L2) as shown in the following figure.

Figure 1. Buffer Sharing

The following steps are performed during DMA buffer sharing:

  1. The V4L2 capture device (client driver) allocates the buffer.
  2. The v4l2src plug-in exports/imports the DMA buffer.
  3. The display driver uses the kernel DMA_BUF framework to locate the buffer location
  4. The display DMA reads the buffer without copying the buffer into kernel memory.

Query Display Configurations

The libdrm library is used to validate if the resolution is supported by the monitor and to query the native resolution of the monitor. The pixel format is configured statically in the device tree.

Media Pipeline Configurations

The video capture pipeline implements a media controller interface that allows you to configure the media pipeline and its sub-devices. The libmediactl and libv4l2subdev libraries provide the following functionality:

  • Enumerate entities, pads, and links
  • Configure sub-devices
    • Set media bus format
    • Set dimensions (width/height)

The Jupyter notebooks set the media bus format and video resolution on each sub-device source and sink pad for the entire media pipeline. The formats between pads that are connected through links must match.