Inspect Float Model Before Quantization - 3.5 English

Vitis AI User Guide (UG1414)
Document ID
UG1414
Release Date
2023-06-29
Version
3.5 English
Vai_q_pytorch provides a function called inspector to help you diagnose neural network (NN) models under different device architectures. The inspector can predict target device assignments based on hardware constraints. The generated inspection report can be used to guide  users to modify or optimize the NN model, greatly reducing the difficulty and time of deployment. It is recommended to inspect float models before quantization.

Take resnet18_quant.py to demonstrate how to edit model code and apply this feature:

  1. Import vai_q_pytorch module
    from pytorch_nndct.apis import Inspector
  2. Create a inspector with target name or fingerprint 
    inspector = Inspector("0x603000b16013831") # by target fingerprint
or
inspector = Inspector("DPUCAHX8L_ISA0_SP") # by target name
  3. Inspect float model
    input = torch.randn([batch_size, 3, 224, 224])
inspector.inspect(model, input)

Run the following command line to inspect model:

python resnet18_quant.py --quant_mode float --inspect

Inspector will display some special messages on screen with special color and special keyword prefix "VAIQ_*" according to the verbose_level setting. Note the messages displayed between "[VAIQ_NOTE]: =>Start to inspect model..." and "[VAIQ_NOTE]: =>Finish inspecting."

If the inspector runs successfully, three important files are usually generated under the output directory "./quantize_result".
inspect_{target}.txt: Target information and all the details of operations in float model
inspect_{target}.svg: If image_format is not None. A visualization of inspection result is generated
inspect_{target}.gv: If image_format is not None. Dot source code of inspetion result is generated
Note:
  • The inspector relies on 'xcompiler' package. In conda env vitis-ai-pytorch in Vitis-AI docker, xcompiler is ready. But if vai_q_pytorch is installed by source code, it needs to install xcompiler in advance.
  • Visualization of inspection results relies on the dot engine. If you don't install dot successfully, set 'image_format = None' when inspecting.
  • If you need more detailed guidance, you can refer to example/jupyter_notebook/inspector/inspector_tutorial.ipynb. Install jupyter notebook in advance. Run the following command:
    jupyter notebook example/jupyter_notebook/inspector/inspector_tutorial.ipynb