Hardware-Aware Quantization Strategy - 3.5 English

Vitis AI User Guide (UG1414)
Document ID
UG1414
Release Date
2023-09-28
Version
3.5 English

The Inspector offers device assignments to operators in the neural network based on the target device, enabling vai_q_pytorch to perform hardware-aware quantization.

Here is the example code in example/resnet18_quant.py:
quantizer = torch_quantizer(quant_mode=quant_mode, 
                            module=model, 
                            input_args=(input), 
                            device=device, 
                            quant_config_file=config_file, 
                            target=target)
For example/resnet18_quant.py, command line to perform hardware-aware calibration:
python resnet18_quant.py --quant_mode calib --target DPUCAHX8L_ISA0_SP
Command line to test hardware-aware quantized model accuracy:
python resnet18_quant.py --quant_mode test --target DPUCAHX8L_ISA0_SP
Command line to deploy the quantized model:
python resnet18_quant.py --quant_mode test --target DPUCAHX8L_ISA0_SP --subset_len 1 --batch_size 1 --deploy