Installing vai_q_pytorch

vai_q_pytorch has GPU and CPU versions. It supports PyTorch version 1.2~2.0 but does not support PyTorch data parallelism. There are two ways to install vai_q_pytorch:

Install Using Docker Containers

Vitis AI provides a Docker container for quantization tools, including vai_q_pytorch. After running a GPU or CPU container, activate the Conda environment, Vitis AI-pytorch.

conda activate vitis-ai-pytorch

Note: In some cases, if you want to install some packages in the Conda environment and encounter permission problems, you can create a separate Conda environment based on vitis-ai-pytorch instead of using vitis-ai-pytorch directly. The pt_pointpillars_kitti_12000_100_10.8G_1.3 model in AMD Model Zoo is an example of this.

A new Conda environment with a specified PyTorch version (1.2~2.0) can be created using the https://github.com/Xilinx/Vitis-AI/blob/v3.5/docker/common/replace_pytorch.sh script. This script does the following:

Clones a Conda environment from Vitis AI-pytorch.
Uninstalls the original PyTorch, Torchvision, and vai_q_pytorch packages.
Installs the specified version of PyTorch, Torchvision.
Re-installs vai_q_pytorch from source code.

The following is the command line to create a new Conda environment with the script:

replace_pytorch.sh new_conda_env_name

Note: Before running the script, check the version of Python, PyTorch, and cuda-toolkit version in the replace_pytorch.sh script and edit them according to your requirement. When choosing PyTorch version and editing the command line, follow the instructions on pytorch official webpage.

Install from the Source Code

vai_q_pytorch is a Python package designed to work as a PyTorch plugin. It is an open source in Vitis_AI_Quantizer. AMD recommends to install vai_q_pytorch in the Conda environment. To do so, follow these steps:

Add the CUDA_HOME environment variable in .bashrc.
For the GPU version, if the CUDA library is installed in /usr/local/cuda, add the following line into .bashrc. If CUDA is in other directory, change the line accordingly.
```
export CUDA_HOME=/usr/local/cuda
```
For the CPU version, remove all CUDA_HOME environment variable setting in your .bashrc. It is recommended to cleanup it in command line of a shell window by running the following command:
```
unset CUDA_HOME
```
Install PyTorch (1.2~2.0) and Torchvision.
The following code takes PyTorch 1.7.1 and torchvision 0.8.2 as an example. You can find detailed instructions for other versions on the PyTorch website.
```
pip install torch==1.7.1 torchvision==0.8.2
```
Install other dependencies.
```
pip install -r requirements.txt
```

Install vai_q_pytorch.

cd ./pytorch_binding 
python setup.py install

Verify the installation.
```
python -c "import pytorch_nndct"
```

Note: If the installed PyTorch version is 1.4 or higher, import pytorch_nndct before importing torch in your script. This is caused by a PyTorch bug in versions prior to 1.4. Refer to PyTorch GitHub issues 28536 and 19668 for details.

import pytorch_nndct
import torch

Installing vai_q_pytorch - 3.5 English

Vitis AI User Guide (UG1414)

Install Using Docker Containers

Install from the Source Code