Running Quantization and Getting the Result - 2.5 English

Vitis AI User Guide (UG1414)

Document ID
Release Date
2.5 English
Note: vai_q_pytorch log messages have special colors and a special keyword prefix, "VAI_Q_*.". vai_q_pytorch log message types include "error", "warning", and "note." Pay attention to vai_q_pytorch log messages to check the flow status.
  1. Run command with "--quant_mode calib" to quantize model.
    python --quant_mode calib --subset_len 200

    When calibrating forward, borrow the float evaluation flow to minimize code change from float script. If you encounter loss and accuracy messages displayed in the end, you can ignore them.

    It is important to control iteration numbers during quantization and evaluation. Generally, 100-1000 images are enough for quantization and the whole validation set is required for evaluation. The iteration numbers can be controlled in the data loading part. In this case, the subset_len argument controls the number of images that are used for network forwarding. If the float evaluation script does not have an argument with a similar role, you must add one.

    If this quantization command runs successfully, two important files are generated in the output directory ./quantize_result.
    Converted vai_q_pytorch format model.
    Quantization steps of tensors. Retain this file for evaluating quantized models.
  2. To evaluate the quantized model, run the following command:
    python --quant_mode test

    The accuracy displayed after the command has executed successfully is the accuracy for the quantized model.

  3. To generate the xmodel for compilation (and onnx format quantized model) , the batch size should be 1. Set subset_len=1 to avoid redundant iterations and run the following command:
    python --quant_mode test --subset_len 1 --batch_size=1 --deploy

    Skip loss and accuracy displayed in the log during running. The xmodel file for the Vitis AI compiler is generated in the output directory, ./quantize_result. It is further used to deploy to the FPGA.

    ResNet_int.xmodel: deployed XIR format model
    ResNet_int.onnx:   deployed onnx format model
    Note: XIR is ready in "vitis-ai-pytorch" conda environment in the Vitis AI docker but if vai_q_pytorch is installed from the source code, you have to install XIR in advance. If XIR is not installed, the xmodel file cannot be generated and the command will return an error. However, you can still check the accuracy in the output log.