For edge DPU, the framework of runtime (called as N2Cube) is shown in the following figure. For Vitis AI release, N2Cube is based on the Xilinx Run time (XRT). For legacy Vivado® based DPU, it interacts with the underlying Linux DPU driver (instead of XRT) for DPU scheduling and resource management.
N2Cube runtime offers a comprehensive set of advanced C++/Python programming interface to flexibly meet the diverse requirements for edge scenarios. Refer to Advanced Programming Interface for more details about edge DPU advanced programming. The highlights for N2Cube are listed as follows:
- Support multi-threading and multi-process DPU application deployment.
- Support multiple models running in parallel and zero-overhead dynamic switching at run-time.
- Automated DPU multi-core scheduling for better workload balancing.
- Optional flexibility to dynamically specify DPU cores affinity over DPU tasks at run-time.
- Priority based DPU tasks scheduling while adhering to DPU cores affinity.
- Optimized memory usage via DPU code and parameter sharing within multi-threading DPU application.
- Easily adaptive to any POSIX-compliant OS or Real-Time Operating System (RTOS) environment, such as QNX, VxWorks, Integrity.
- Ease-of-use capabilities for DPU debugging and performance profiling.
Currently, N2Cube officially supports three operating environments, including Linux, Xilinx XRT, and BlackBerry QNX RTOS. You can contact the Xilinx representatives to acquire Vitis AI package for QNX or to port N2Cube to other third party RTOSs.