vai_q_pytorch is designed to work as a Pytorch plugin. Xilinx provides the simplest APIs to introduce the FPGA-friendly quantization feature. vai_q_pytorch package is already installed in docker image "vitis-ai-pytorch" environment. A Resnet18 example is in the open-source repo. Use the following steps to quantize a Resnet18 model.
- Prepare a floating-point model for Resnet-18. You can download one from Pytorch official
wget https://download.pytorch.org/models/resnet18-5c106cde.pth -O resnet18.pth
- Prepare the calibration dataset used by vai_q_pytorch. You can download 100 to 1000 images of ImageNet dataset from http://academictorrents.com/collection/imagenet-2012 or http://www.image-net.org/download.php
- Copy float model
resnet18.pth, example code
resnet18_quant.py, and calibration images to docker image and modify default
- Activate the Pytorch running
conda activate vitis-ai-pytorch
- Evaluate float
python resnet18_quant.py --quant_mode 0
- Quantize, using a subset (200 images) of validation data for calibration.
Because it is in quantize calibration process, the displayed loss and accuracy are
python resnet18_quant.py --quant_mode 1 --subset_len 200
- Evaluate quantized model and generate xmodel file for compiler.
python resnet18_quant.py --quant_mode 2
ResNet_int.xmodelfile will be generated under folder
quantize_result. It could be fed to VAI compiler for following compilation process.