Frameskip Support for the VCU Encoder - 2023.1 English

H.264/H.265 Video Codec Unit v1.2 Solutions LogiCORE IP Product Guide (PG252)

Document ID
PG252
Release Date
2023-05-16
Version
2023.1 English

The VCU encoder currently supports the frame skip feature to strictly achieve the required target bitrate. When an encoded picture is too large and exceeds the CPB buffer size, the picture is discarded and replaced by a picture with all MB/CTB encoded as « Skip ». This feature is useful especially at low bitrates when the encoder must maintain a strict target-bitrate irrespective of video-complexity.

You can also control the maximum number of consecutive skipped frames. However, if the maximum number of consecutive skipped frames is set too low, the output bitstream may not achieve the target bitrate. So, you may need to experiment to find a balance between the achieved bitrate and number of actual encoded frames.

Constraints

  • This only applies if the reference frame has already been encoded. The first frame is encoded irrespective of the buffer overflow. Only subsequent pictures are discarded if actual frame size is exceeded.

    If the CPB is very small and the video is complex, all frames might be discarded.

  • HRD compliance for dynamic commands is not guaranteed when the bitrate or fps change. Skip frames can be added or missed when those parameters change.
    • The back-and-forth visual effect can appear when the value of NumB is greater than 1 and enable-skip is set to true.
    • In the dual pass mode, frame skip can only be used on the second pass.
    • Inserting the frame skip in GOP pattern with more than one B-frame can have back and forth visual effect. NumB >1 is not recommended.
    • Avoid frame skip in adaptive GOP feature.

Not Supported

  • Frameskip is not supported in the low latency (--slicelat) mode.
  • Frameskip is not supported in Pyramidal GOP.

Control Software API

  • The AL_RC_OPT_ENABLE_SKIP bit is used in "eOptions" member variable of AL_TRCParam structure, which is encapsulated in AL_TEncChanParam structure to enable the frameskip feature.
  • The nal_ref_idc flag can be used to detect skip frame as all the skip frames are not used for reference, which means that if the slice type is P and nal_ref_idc = 0, then that frame is a skip frame in the IPPPPPP GOP mode.
  • The frame skip flag feature notifies user applications about which frame has been skipped to allow users to drop/do some extra processing on skipped frames.
  • A new bSkipped flag has been added into the buffer metadata. When an encoded output buffer is ready, user applications can extract the buffer metadata from the stream buffer using the control software API, AL_Buffer_GetMetaData(). The boolean flag bSkipped indicates whether or not that particular buffer was skipped.
  • A new uMaxConsecSkip member variable of the AL_TRCParam structure is used to set the maximum number of consecutive skipped frames.

OMX API

  • New SetParam Index: OMX_ALG_IndexParamVideoSkipFrame
  • Data structure: OMX_ALG_VIDEO_PARAM_SKIP_FRAME
  • bEnableSkipFrame flag of OMX_ALG_VIDEO_PARAM_SKIP_FRAME structure is used to indicate if skip frame should be enabled.
  • nMaxConsecutiveSkipFrame of OMX_ALG_VIDEO_PARAM_SKIP_FRAME structure is used to indicate the maximum number of consecutive skipped frames.

For enabling frameskip, enable the "EnableSkip" parameter in cfg file:

EnableSkip = TRUE

For setting the maximum number of consecutive skipped frames, set the “MaxConsecutiveSkip” parameter in cfg file:

MaxConsecutiveSkip = 5

Use the following commands:

  • ctrlsw_encoder -cfg test.cfg
  • ctrlsw_encoder -cfg test.cfg -i test.yuv --input-width 1280 --input-height 720 -o test.avc --print-picture-type