Deployment - 2.5 English

Vitis AI User Guide (UG1414)

Document ID
Release Date
2.5 English

In this section, we will deploy the pytorch model with custom op in graph_runner APIs. For graph_runner APIs, it supports both C++ and Python. You can refer to the graph_runner samples in Vitis-AI/examples/Vitis-AI-Library/samples/graph_runner.

  1. Create a new directory to hold your test code.
  2. Create source file and implement the following functions for this sample.
    1. Parameter parse and initialize
    2. Preprocess
    3. Model_run
    4. Postprocess
    The basic flow of this sample is shown in the following image.
    Figure 1. The basic flow of this sample
  3. Build the program.
    1. If your project is simple, such as only one .cpp source file, you can copy any existing from Vitis-AI/examples/Vitis-AI-Library/samples/graph_runner and modify it accordingly. Then, run the following command to build the program.
      cd <your sample folder>
      The following figure shows the of resnet50_graph_runner sample.
    2. If your project is more complex, such as this sample, it's better to use CMakeLists.txt for easy compiling. For more details about CMakeLists.txt, refer to Vitis-AI/examples/Custom_OP_Demo/pytorch_example/deployment/cpp/pointpillars_graph_runner/CMakeLists.txt
      Then, run the following command to build the program.
      cd <your sample folder>
      mkdir build
      cd build
      cmake ..

    After a successful compilation, the executable program sample_pointpillars_graph_runner will be generated under <your sample folder>

  4. Test the program
    Before you test the program, please copy the xmodel, test image, custom op library and executable program sample_pointpillars_graph_runner to the board. Put the custom op library under /usr/lib. Then, run the following command to do the test.
    ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin

    The following shows the running result of this sample.

    root@xilinx-zcu102-2021_2:~/pointpillars_graph_runner# ./sample_pointpillars_graph_runner pointpillars_op.xmodel sample_pointpillars.bin
    WARNING: Logging before InitGoogleLogging() is written to STDERR
    W1202 05:59:20.517452 1307 tool_function.cpp:177] [UNILOG][WARNING] The operator named VoxelNet__VoxelNet_input_4, type: PPScatterV2, is not defined in XIR. XIR creates the definition of this operator automatically. You should specify the shape and the data_type of the output tensor of this operation by set_attr("shape", std::vector) and set_attr("data_type", std::string)
    result: 0
    0 18.541065 3.999999 -1.732742 1.703191 4.419279 1.465484 1.679375 0.880797
    0 34.522400 1.505865 -1.515198 1.503061 3.550991 1.420396 1.710625 0.851953
    0 10.917599 4.705865 -1.622433 1.650789 4.350764 1.634866 1.632500 0.851953
    1 21.338514 -2.400001 -1.681677 0.600000 1.963422 1.784916 4.742843 0.777300
    0 57.891731 -4.188268 -1.536627 1.575194 3.780010 1.512004 2.007500 0.679179
If you want to profile the sample of custom op, use the environment variable DEEPHI_PROFILING=1, as shown below.
env DEEPHI_PROFILING=1 ./sample_pointpillars_graph_runner ./pointpillars_full_customer_op.xmodel sample_pointpillars.bin
The profiling result is shown below.
I1130 01:29:53.038476 15571 cpu_task.cpp:163] CPU_UPDATE_INPUT : 5us
I1130 01:29:53.038684 15571 cpu_task.cpp:166] CPU_UPDATE_OUTPUT : 55us
I1130 01:29:53.038872 15571 cpu_task.cpp:169] CPU_SYNC_FOR_READ : 46us
I1130 01:29:53.039050 15571 cpu_task.cpp:181] CPU_OP_EXEC : 32us
I1130 01:29:53.039232 15571 cpu_task.cpp:181] CPU_OP_EXEC : 36us
I1130 01:29:53.039597 15571 cpu_task.cpp:181] CPU_OP_EXEC : 232us
I1130 01:29:53.066352 15571 cpu_task.cpp:181] CPU_OP_EXEC : 26575us
I1130 01:29:53.066745 15571 cpu_task.cpp:195] CPU_SYNC_FOR_WRITE : 1us