Running Quantization and Getting the Result

Running Quantization and Getting the Result - 1.4.1 English

Vitis AI User Guide (UG1414)

Document ID

UG1414

Release Date

2021-12-13

Version

1.4.1 English

Note: vai_q_pytorch log messages have special colors and a special keyword, "NNDCT." "NNDCT" is an internal project name and you can change it later. vai_q_pytorch log message types include "error", "warning", and "note." Pay attention to vai_q_pytorch log messages to check the flow status.

Run command with "--quant_mode calib" to quantize model.
```
python resnet18_quant.py --quant_mode calib --subset_len 200
```
When calibrating forward, borrow the float evaluation flow to minimize code change from float script. If there are loss and accuracy messages displayed in the end, you can ignore them. Note the colorful log messages with the special keyword, "NNDCT".

It is important to control iteration numbers during quantization and evaluation. Generally, 100-1000 images are enough for quantization and the whole validation set is required for evaluation. The iteration numbers can be controlled in the data loading part. In this case, the subset_len argument controls the number of images that are used for network forwarding. If the float evaluation script does not have an argument with a similar role, you must add one.

If this quantization command runs successfully, two important files are generated in the output directory ./quantize_result.

ResNet.py

Converted vai_q_pytorch format model.

Quant_info.json

Quantization steps of tensors. Retain this file for evaluating quantized models.
To evaluate the quantized model, run the following command:
```
python resnet18_quant.py --quant_mode test
```
The accuracy displayed after the command has executed successfully is the right accuracy for the quantized model.
To generate the xmodel for compilation, the batch size should be 1. Set subset_len=1 to avoid redundant iterations and run the following command:
```
python resnet18_quant.py --quant_mode test --subset_len 1 --batch_size=1 --deploy
```
Skip loss and accuracy displayed in the log during running. The xmodel file for the Vitis AI compiler is generated in the output directory, ./quantize_result. It is further used to deploy to the FPGA.
```
ResNet_int.xmodel: deployed model
```
Note: XIR is ready in "vitis-ai-pytorch" conda environment in the Vitis AI docker but if vai_q_pytorch is installed from the source code, you have to install XIR in advance. If XIR is not installed, the xmodel file cannot be generated and the command will return an error. However, you can still check the accuracy in the output log.