To generate the quantized inference model and reference result, follow these steps:
- Generate the quantized inference model by running the following commands.
vai_q_caffe quantize -model float/test_quantize.prototxt \ -weights float/trainval.caffemodel \ -output_dir quantize_model \ -keep_fixed_neuron \ 2>&1 | tee ./log/quantize.log
The following files are generated in the
- Generate the reference result by running the following
DECENT_DEBUG=5 vai_q_caffe test -model quantize_model/dump.prototxt \ -weights quantize_model/quantize_train_test.caffemodel \ -test_iter 1 \ 2>&1 | tee ./log/dump.log
This creates the
dump_gpufolder and files as shown in the following figure.
- Generate the DPU xmodel by running the following command, for example,
vai_c_caffe --prototxt quantize_model/deploy.prototxt \ --caffemodel quantize_model/deploy.caffemodel \ --arch /opt/vitis_ai/compiler/arch/DPUCAHX8H/U50LV/arch.json \ --output_dir compile_model \ --net_name resnet50
- Generate the DPU inference result by running the following command.
env XLNX_ENABLE_DUMP=1 XLNX_ENABLE_DEBUG_MODE=1 \ xdputil run ./compile_model/resnet50.xmodel \ ./dump_gpu/data.bin 2>result.log 1>&2
The DPU inference result and the comparing result
result.logare generated if this command runs successfully. The DPU inference results are located in the
- Crosscheck the reference result and the DPU inference result.
The crosscheck mechanism is to first make sure input(s) to one layer is identical to reference and then the output(s) is identical too. This can be done with commands like
cmp. If two files are identical,
cmpwill return nothing in the command line.
- Check the input of DPU and GPU to ensure they use the same input data.
- Use the
xdputiltool to generate a picture for displaying the network structure.
Usage: xdputil xmodel <xmodel> -s <svg>Note: To install the required library, execute the following command in the Vitis AI docker environment.
sudo apt-get install graphviz
The following figure is part of the ResNet50 model structure generated by
- View the xmodel structure image and find out the last layer name of the
model.Recommended: Check the last layer first. If the crosscheck of the last layer is successful, then the crosscheck for all the layers will pass and there is no need crosscheck each layers individually.
For this model, the name of the last layer is `subgraph_fc1000_fixed_(fix2float)`.
Note: For the layers that have multiple input or output (e.g.,
- Search the keyword
dump. You will find the reference result file
dump_gpuand DPU inference result
- Diff the two files.
If the crosscheck for the last layer fails, perform the crosscheck from the first layer until you find the layer where the crosscheck fails.
res2a_branch1), input correctness should be checked before verifying the output.
- Search the keyword
- Submit the files to Xilinx if the DPU
cross check fails.
If a certain layer proves to be wrong on the DPU, prepare the following files as one package for further analysis and send it to Xilinx with a detailed description.
- Float model and prototxt file
- Quantized model, such as deploy.caffemodel, deploy.prototxt, quantize_train_test.caffemodel, and quantize_train_test.prototxt.