- Run command with "--quant_mode calib" to quantize
python resnet18_quant.py --quant_mode calib --subset_len 200
When calibrating forward, borrow the float evaluation flow to minimize code change from float script. If there are loss and accuracy messages displayed in the end, you can ignore them. Note the colorful log messages with the special keyword, "NNDCT".
It is important to control iteration numbers during quantization and evaluation. Generally, 100-1000 images are enough for quantization and the whole validation set is required for evaluation. The iteration numbers can be controlled in the data loading part. In this case, the
subset_lenargument controls the number of images that are used for network forwarding. If the float evaluation script does not have an argument with a similar role, you must add one.
If this quantization command runs successfully, two important files are generated in the output directory ./quantize_result.
- Converted vai_q_pytorch format model.
- Quantization steps of tensors. Retain this file for evaluating quantized models.
- To evaluate the quantized model, run the following
python resnet18_quant.py --quant_mode test
The accuracy displayed after the command has executed successfully is the right accuracy for the quantized model.
- To generate the xmodel for compilation, the batch size should be 1. Set
subset_len=1to avoid redundant iterations and run the following command:
python resnet18_quant.py --quant_mode test --subset_len 1 --batch_size=1 --deploy
Skip loss and accuracy displayed in the log during running. The xmodel file for the Vitis AI compiler is generated in the output directory, ./quantize_result. It is further used to deploy to the FPGA.
ResNet_int.xmodel: deployed modelNote: XIR is ready in "vitis-ai-pytorch" conda environment in the Vitis AI docker but if vai_q_pytorch is installed from the source code, you have to install XIR in advance. If XIR is not installed, the xmodel file cannot be generated and the command will return an error. However, you can still check the accuracy in the output log.
Note: vai_q_pytorch log messages have special colors and a special keyword, "NNDCT." "NNDCT" is an internal project name and you can change it later. vai_q_pytorch log message types include "error", "warning", and "note." Pay attention to vai_q_pytorch log messages to check the flow status.