Hardware-Aware Quantization Strategy - 3.0 English

Vitis AI User Guide (UG1414)
Document ID
UG1414
Release Date
2023-02-24
Version
3.0 English

Inspector provides device assignments to operators in the neural network based on the target device. vai_q_pytorch can use the power of inspector to perform hardware-aware quantization.

Example code in example/resnet18_quant.py:
quantizer = torch_quantizer(quant_mode=quant_mode, 
                            module=model, 
                            input_args=(input), 
                            device=device, 
                            quant_config_file=config_file, 
                            target=target)
For example/resnet18_quant.py, command line to do hardware-aware calibration:
python resnet18_quant.py --quant_mode calib --target DPUCAHX8L_ISA0_SP
command line to test hardware-aware quantized model accuracy:
python resnet18_quant.py --quant_mode test --target DPUCAHX8L_ISA0_SP
command line to deploy quantized model:
python resnet18_quant.py --quant_mode test --target DPUCAHX8L_ISA0_SP --subset_len 1 --batch_size 1 --deploy