Description - 2023.1 English

Zynq UltraScale+ MPSoC ZCU106 Video Codec Unit Targeted Reference Design User Guide (UG1250)

Document ID
UG1250
Release Date
2023-05-16
Version
2023.1 English

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple playback and audio/video streaming to complex audio (mixing) and video processing.

GStreamer uses a plug-in architecture which makes the most of GStreamer functionality implemented as shared libraries. The GStreamer base functionality contains functions for registering and loading plug-ins and for providing the fundamentals of all classes in the form of base classes. Plug-in libraries get dynamically loaded to support a wide spectrum of codecs, container formats, and input/output drivers.

Table: GStreamer Plug-ins describes the plug-ins used in the GStreamer interface library.

Table 3-7: GStreamer Plug-ins

Plug-in

Description

v4l2src

v4l2src can be used to capture video from V4L2 devices such as AMD HDMI-RX and TPG.

Example pipeline:

gst-launch-1.0 v4l2src ! kmssink

This pipeline shows the video captured from a /dev/video0 and rendered on a display unit.

kmssink

The kmssink is a simple video sink that renders raw video frames directly in a plane of a DRM device.

Example pipeline:

gst-launch-1.0 v4l2src ! “video/x-raw, format=NV12, width=3840, height=2160” ! kmssink

omxh26xdec

Decoder omxh26xdec is a hardware-accelerated video decoder that decodes encoded video frames.

Example pipeline:

gst-launch-1.0 filesrc location=/media/card/abc.mp4 ! qtdemux ! h26xparse ! omxh26xdec ! kmssink

This pipeline shows an .mp4 multiplexed file where the encoded format is h26x encoded video.

Note: Use omxh264dec for H264 decoding and omxh265dec for H265 decoding.

omxh26xenc

Encoder omxh26xenc is a hardware-accelerated video encoder that encodes raw video frames.

Example pipeline:

gst-launch-1.0 v4l2src ! omxh26xenc ! filesink location=out.h26x

This pipeline shows the video captured from a V4L2 device that delivers raw data. The data is encoded to the h26x encoded video type and stored to a file.

Note: Use omxh264enc for H264 encoding and omxh265enc for H265 encoding.

alsasrc

The alsasrc plug-in can be used to capture audio from audio devices such as AMD HDMI-RX.

Example pipeline:

gst-launch-1.0 alsasrc device=hw:1,1 ! queue ! audioconvert ! audioresample ! audio/x-raw, rate=48000, channels=2, format=S24_32LE ! alsasink device="hw:1,0"

This pipeline shows the audio captured from an ALSA source and plays on an ALSA sink.

alsasink

The alsasink is a simple audio sink that plays raw audio frames.

Example pipeline:

gst-launch-1.0 alsasrc device=hw:1,1 ! queue ! audioconvert ! audioresample ! audio/x-raw, rate=48000, channels=2, format=S24_32LE ! alsasink device="hw:1,0"

This pipeline shows the audio captured from the ALSA source and plays on an ALSA sink.

faad

Decoder faad is an audio decoder that decodes encoded audio frames.

Example pipeline:

gst-launch-1.0 filesrc location=out.ts ! tsdemux ! aacparse ! faad ! audioconvert ! audioresample ! audio/x-raw,rate=48000,channnels=2, format=S24_32LE ! alsasink device="hw:1,0"

This pipeline shows a .ts multiplexed file where the encoded format is aac encoded audio. The data is decoded and played on an ALSA sink device.

faac

Encoder faac is an audio encoder that encodes raw audio frames.

Example pipeline:

gst-launch-1.0 alsasrc device=hw:1,1 num-buffers=500 ! audio/x-raw, format=S24_32LE, rate=48000 ,channels=2 ! queue ! audioconvert ! audioresample ! faac ! aacparse ! mpegtsmux ! filesink location=out.ts

This pipeline shows the audio captured from an ALSA device that delivers raw data. The data is encoded to aac format and stored to a file.

xilinxscd

xilinxscd is hardware-accelerated IP that enables detection of scene change in a video stream. This plugin generates upstream events whenever there is scene change in an incoming video stream so the encoder can insert an Intra frame to improve video quality.

Example pipeline:

gst-launch-1.0 -v v4l2src ! video/x-raw, width=3840, height=2160, format=NV12, framerate=60/1 ! xilinxscd io-mode=5 ! omxh26xenc ! filesink location=/run/out.h26x

This pipeline shows the video captured from a V4L2 device that delivers raw data. This raw data is passed through the xilinxscd plugin which analyzes the stream in runtime and provides an event to the encoder whether any scene change is detected in a video stream or not. The encoder uses this information to insert an I-frame in an encoded bit-stream.

Use omxh264enc for H264 encoding and omxh265enc for H265 encoding.

appsrc

The appsrc element can be used by applications to insert data into a GStreamer pipeline. Unlike most GStreamer elements, appsrc provides external API functions.

appsink

appsink is a sink plugin that supports many different methods, enabling the application to manage the GStreamer data in a pipeline. Unlike most GStreamer elements, appsink provides external API functions.