Deployment - 3.5 English

This section describes deploying the PyTorch model with custom op in graph_runner APIs. graph_runner APIs support both C++ and Python. You can refer to the graph_runner samples in Vitis-AI/examples/vai_library/samples/graph_runner:

1. Create a new directory for your test code.
2. Create a source file and implement the following functions for this sample:
   1. Parameter parse and initialize
   2. Preprocess
   3. Model_run
   4. Postprocess
   The basic flow of this sample is shown in the following image.
   Figure 1. The basic flow of this sample
3. Build the program.
   1. If your project is relatively simpler, consisting of a single or few .cpp source files, you can easily set up the build process by duplicating any existing build.sh script from Vitis-AI/examples/vai_library/samples/graph_runner and adapting it to your specific needs. After you have customized the build.sh script, execute the following command to build your program:

      cd <your sample folder>
      bash build.sh

      The following figure shows the build.sh of the resnet50_graph_runner sample.
      
   2. If your project is more complex, such as this sample, using CMakeLists.txt for easy compiling is better. For more details about CMakeLists.txt, refer to Vitis-AI/examples/custom_operator/pytorch_example/deployment/cpp/pointpillars_graph_runner/CMakeLists.txt
      Then, run the following command to build the program:

      cd <your sample folder>
      mkdir build
      cd build
      cmake ..
      make

   After a successful compilation, the executable program sample_pointpillars_graph_runner will be generated under <your sample folder>

4. Test the program.
   Before you test the program, copy the XMODEL, test image, custom op library libvart_op_imp_PPScatterV2.so and the sample_pointpillars_graph_runner executable program to the board. Place the custom op library in /usr/lib. Then, run the following command to do the test:

   ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin

   The following is a running result snapshot of this sample:

   root@xilinx-zcu102-2021_2:~/pointpillars_graph_runner# ./sample_pointpillars_graph_runner pointpillars_op.xmodel sample_pointpillars.bin
   WARNING: Logging before InitGoogleLogging() is written to STDERR
   W1202 05:59:20.517452 1307 tool_function.cpp:177] [UNILOG][WARNING] The operator named VoxelNet__VoxelNet_input_4, type: PPScatterV2, is not defined in XIR. XIR creates the definition of this operator automatically. You should specify the shape and the data_type of the output tensor of this operation by set_attr("shape", std::vector) and set_attr("data_type", std::string)
   result: 0
   0 18.541065 3.999999 -1.732742 1.703191 4.419279 1.465484 1.679375 0.880797
   0 34.522400 1.505865 -1.515198 1.503061 3.550991 1.420396 1.710625 0.851953
   0 10.917599 4.705865 -1.622433 1.650789 4.350764 1.634866 1.632500 0.851953
   1 21.338514 -2.400001 -1.681677 0.600000 1.963422 1.784916 4.742843 0.777300
   0 57.891731 -4.188268 -1.536627 1.575194 3.780010 1.512004 2.007500 0.679179

env DEEPHI_PROFILING=1 ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin

I1130 01:29:53.038476 15571 cpu_task.cpp:163] CPU_UPDATE_INPUT : 5us
I1130 01:29:53.038684 15571 cpu_task.cpp:166] CPU_UPDATE_OUTPUT : 55us
I1130 01:29:53.038872 15571 cpu_task.cpp:169] CPU_SYNC_FOR_READ : 46us
I1130 01:29:53.039050 15571 cpu_task.cpp:181] CPU_OP_EXEC : 32us
I1130 01:29:53.039232 15571 cpu_task.cpp:181] CPU_OP_EXEC : 36us
I1130 01:29:53.039597 15571 cpu_task.cpp:181] CPU_OP_EXEC : 232us
I1130 01:29:53.066352 15571 cpu_task.cpp:181] CPU_OP_EXEC : 26575us
I1130 01:29:53.066745 15571 cpu_task.cpp:195] CPU_SYNC_FOR_WRITE : 1us