The VCU encoder currently supports Region of Interest (ROI) encoding which allows you to define several independent or overlapped regions within a frame. The ROI encoding tags regions in a video frame to be encoded with user supplied quality (high, medium, low, and dont-care) relative to the picture background (untagged region). An example ROI defined in a frame is shown in the following figure .
You can provide the region of interest (ROI) location (top, left) in pixels and width and height in pixels along with quality index. Multiple and overlapped ROI regions within a frame are supported. The sample GStreamer application only adds one ROI region but you can attach multiple ROI meta data properties to the buffer.
The input format is:
ROI:<frame number>:<top>x<left>:<width>x<height>:<ROI type>
A sample GStreamer application command line option to encode video with ROI region attached to specific frame number at 100 is:
zynqmp_vcu_encode -w 3840 -h 2160 -e avc -o /run/op.h264 -i /run/input.yuv -d ROI:100:1280x360:1220x1500:high
Custom ROI Delta-QP
The new custom ROI delta-qp feature allows you to provide your own delta-qp value for any region of interest to modify the quality level.
The following dynamic command has been added to the zynqmp_vcu_encode application:
This creates a new gstreamer meta structure:
roi-by-value/omx-alg. Using this meta structure, the
gstreamer fills out the new OMX config,
OMX_ALG_VIDEO_CONFIG_REGION_OF_INTEREST_BY_VALUE, to send the
parameters to the VCU.
The following is an example pipeline:
zynqmp_vcu_encode -w 1920 -h 1080 -e avc -f 60 -c 2 -g 30 -o test.h264 -i test.yuv -d ROI_BY_VALUE:0:1200x300:200x200:10 -q 1
You can set ROI live streaming dynamically using the GStreamer and control software.
As showcased in example application code, use the following two APIs to set ROI region and quality for each frame: https://github.com/Xilinx/gstreamer/blob/xlnx-rebase-v1.20.5/subprojects/gst-omx/examples/zynqultrascaleplus/test-vcu-encode.c (line number #501).
The control software exports a few APIs to provide ROI information to the encoder per frame. Use these APIs to add ROIs in live streaming.
AL_RoiMngr_Create() called during encoder creation.
AL_RoiMngr_Clear(), AL_Roi_Mngr_AddROI(), and AL_RoiMngr_FillBuff() are called for each frame to provide ROI information to encoder.
Refer to the Doxygen output for more details on APIs.