PointPainting - 2.0 English

Vitis AI Library User Guide (UG1354)

Document ID
UG1354
Release Date
2022-01-20
Version
2.0 English

For AD/ADAS systems, sensor-fusion algorithms play a significant role in providing high-quality perception and increasing the safety level for driving. PointPainting provides a sensor-fusion framework that takes advantage of 2D semantic segmentation and 3D object detection models. First, a network is applied to the camera images for semantic segmentation. Based on the semantic information and calibration information (on camera and LiDAR), the LiDAR point clouds are projected to the images and fused with the semantic information to get the painted point clouds. Finally, the painted point clouds are consumed by the 3D object detector to achieve better perception.

Figure 1. PointPainting Example

The following table lists the PointPainting models supported by the Vitis AI library.

Table 1. PointPainting Models
No Model Name Framework
1 pointpainting_nuscenes_40000_64_0_pt PyTorch
2 pointpainting_nuscenes_40000_64_1_pt PyTorch
3 semanticfpn_nuimage_576_320_pt PyTorch