Software stacks can be quite complex with many layers and different standards and APIs. On the kernel side, the display and graphics portions are split with each having their own APIs. However, both are commonly referred to as a single framework, DRM/KMS. This split is advantageous, especially for SoCs that often have dedicated hardware blocks for display. The display pipeline driver responsible for interfacing with the display uses the kernel mode setting (KMS) API. It is accessed from user space through a single device node. The display driver stack is shown in the following figure.
Software Stack
The APU Linux multimedia software stack is divided into an application layer and a platform layer. The application layer is implemented in the Linux user-space whereas the platform layer contains middleware (user-space libraries) and operating system (OS) components (kernel-space drivers). This chapter focuses on the application layer implemented in the user space.
The Jupiter Notebook application, based on the GStreamer, demonstrates features of the TRD. The following table describes the software components.
| Component | Description |
|---|---|
| Kernel drivers | This layer contains the kernel drivers for HDMI, IMX274 sensor driver, MIPI CSI-2 RX Subsystem, Xilinx Video Demosaic, Xilinx Video Gamma LUT, VPSS Color Space Converter (CSC), Xilinx Video Processing Subsystem (VPSS Only configuration, 2X configuration), HDMI TX Subsystem. |
| User space libraries | User space libraries include the media and v4l2 lib for the video pipeline, GStreamer libraries, libdrm for the DRM device, Vp9 Softcodec, and python packages to support Jupyter notebooks. |
| GStreamer framework | GStreamer is the cross-platform/open source multimedia framework, and provides the infrastructure to integrate multiple multimedia components and create pipelines. Various GStreamer plug-ins are used for input, filter, and display components. |
Jupyter notebooks provide the necessary interfaces to execute the supported use cases in this TRD.