In this section, we will deploy the pytorch model with custom op in graph_runner APIs. For graph_runner APIs, it supports both C++ and Python. You can refer to the graph_runner samples in Vitis-AI/examples/Vitis-AI-Library/samples/graph_runner.
- Create a new directory to hold your test code.
- Create source file and implement the following functions for this sample.
Figure 1. The basic flow of this sample
- Parameter parse and initialize
- Build the program.
- If your project is simple, such as only one .cpp source file, you can copy
any existing build.sh from Vitis-AI/examples/Vitis-AI-Library/samples/graph_runner
and modify it accordingly. Then, run the following command to build the
The following figure shows the build.sh of resnet50_graph_runner sample.
cd <your sample folder> bash build.sh
- If your project is more complex, such as this sample, it's better to use
CMakeLists.txt for easy compiling. For more details about
CMakeLists.txt, refer to Vitis-AI/examples/Custom_OP_Demo/pytorch_example/deployment/cpp/pointpillars_graph_runner/CMakeLists.txt
Then, run the following command to build the program.
cd <your sample folder> mkdir build cd build cmake .. make
After a successful compilation, the executable program sample_pointpillars_graph_runner will be generated under <your sample folder>
- If your project is simple, such as only one .cpp source file, you can copy any existing build.sh from Vitis-AI/examples/Vitis-AI-Library/samples/graph_runner and modify it accordingly. Then, run the following command to build the program.
- Test the programBefore you test the program, please copy the xmodel, test image, custom op library libvart_op_imp_PPScatterV2.so and executable program sample_pointpillars_graph_runner to the board. Put the custom op library under /usr/lib. Then, run the following command to do the test.
./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin
The following shows the running result of this sample.
root@xilinx-zcu102-2021_2:~/pointpillars_graph_runner# ./sample_pointpillars_graph_runner pointpillars_op.xmodel sample_pointpillars.bin WARNING: Logging before InitGoogleLogging() is written to STDERR W1202 05:59:20.517452 1307 tool_function.cpp:177] [UNILOG][WARNING] The operator named VoxelNet__VoxelNet_input_4, type: PPScatterV2, is not defined in XIR. XIR creates the definition of this operator automatically. You should specify the shape and the data_type of the output tensor of this operation by set_attr("shape", std::vector) and set_attr("data_type", std::string) result: 0 0 18.541065 3.999999 -1.732742 1.703191 4.419279 1.465484 1.679375 0.880797 0 34.522400 1.505865 -1.515198 1.503061 3.550991 1.420396 1.710625 0.851953 0 10.917599 4.705865 -1.622433 1.650789 4.350764 1.634866 1.632500 0.851953 1 21.338514 -2.400001 -1.681677 0.600000 1.963422 1.784916 4.742843 0.777300 0 57.891731 -4.188268 -1.536627 1.575194 3.780010 1.512004 2.007500 0.679179
env DEEPHI_PROFILING=1 ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin
I1130 01:29:53.038476 15571 cpu_task.cpp:163] CPU_UPDATE_INPUT : 5us I1130 01:29:53.038684 15571 cpu_task.cpp:166] CPU_UPDATE_OUTPUT : 55us I1130 01:29:53.038872 15571 cpu_task.cpp:169] CPU_SYNC_FOR_READ : 46us I1130 01:29:53.039050 15571 cpu_task.cpp:181] CPU_OP_EXEC : 32us I1130 01:29:53.039232 15571 cpu_task.cpp:181] CPU_OP_EXEC : 36us I1130 01:29:53.039597 15571 cpu_task.cpp:181] CPU_OP_EXEC : 232us I1130 01:29:53.066352 15571 cpu_task.cpp:181] CPU_OP_EXEC : 26575us I1130 01:29:53.066745 15571 cpu_task.cpp:195] CPU_SYNC_FOR_WRITE : 1us