1 Introduction
- PTQ: Post-training Quantization, refers to the process of quantization of floating point model directly through tools based on some calibration data after the completion of training. Compared with QAT, PTQ is simpler.
- QAT: Quantization aware training refers to the process of inserting pseudo-quantization nodes into the model for quantization training after the completion of floating-point model training. Compared with PTQ, the accuracy of QAT is more guaranteed. Horizon provides local manual installation and Docker container for both PTQ and QAT quantization schemes. It is recommended that you prefer the easy-to-use Docker. About Docker environment deployment, Welcome to refer to the user manual/Docker mirror deployment environment (https://developer.horizon.ai/api/v1/fileData/horizon_j5_open_explorer_cn_doc/oe_mapper/source/env _install/env_deploy.html#docker). Of course, for the convenience of some users who cannot use the Docker provided by Horizon, this article will cover installing the Horizon QAT environment without affecting the local global Python environment. This article introduces the process of deploying the QAT local environment based on the Linux system. All the dependencies used are Python3.8.
2 conda environment creation and use
Conda is an open source package management system and environment management system running on Windows, macOS, and Linux. Conda quickly installs, runs, and updates packages and their dependencies, making it easy to create, save, load, and switch environments on your local computer. This article will create a python virtual environment based on conda. Before creating a virtual environment, please make sure that your machine has been successfully installed [Anadonca] (https://www.anaconda.com/download). 2.1 Create a python virtual environment
#horizon_qat is the name of the newly created virtual environment
# Specifies the version of the virtual environment to be created as [python=3.8.10]. If not specified, the default is the python version when anaconda was installed
conda create --name horizon_qat python=3.8.10
2.2 Activation and shutdown of conda virtual environment
# View your local conda virtual environment
conda env list
# Activate the specified virtual environment
conda activate horizon_qat
# Close the virtual environment
conda deactivte
3 Dependencies required to install QAT
To realize the horizon QAT function, only four whl packages need to be manually installed, namely horizon_plugin_pytorch.whl, hbdk.whl, torch.whl, and torchvision.whl. The first two are developed by Horizon. All four whl packages are available from the ddk/package/host/ai_toolchain directory in the Horizon Toolchain development kit OpenExplorer. This article uses horizon_j5_open_explorer_v1.1.45-py38_20230323 as an example to describe the four whl packages that need to be manually installed as follows:
Installation library name
Installation requirements
horizon_plugin_PyTortor-1.4.4 +cu111.torch1102-cp38-cp38-linux_x86_64.whl
cuda11.1,torch1.10.2+cuda-11.1,python3.8
hbdk-3.43.4-cp38-cp38-linux_x86_64.whl
python3.8
torch-1.10.2+cu111-cp38-cp38-linux_x86_64.whl
cuda11.1,python3.8
torchvision-0.11.3+cu111-cp38-cp38-linux_x86_64.whl
cuda11.1,python3.8
Note that Python versions correspond Execute the following instructions in order to install:
pip3 install torch-1.10.2+cu111-cp38-cp38-linux_x86_64.whl
pip3 install torchvision-0.11.3+cu111-cp38-cp38-linux_x86_64.whl
pip3 install hbdk-3.43.4-cp38-cp38-linux_x86_64.whl
pip3 install horizon_plugin_PyTortor-1.4.4 +cu111.torch1102-cp38-cp38-linux_x86_64.whl
After the installation is complete, run the following command on the CLI. If the following information is displayed, the environment is successfully installed:
# Enter the python command line
python3
# Import horizon_plugin_pytorch and print the version
import horizon_plugin_pytorch
print(horizon_plugin_pytorch.__version__)
# If the installation is successful, the following message will be printed:
# 1.4.4 + cu111 torch1102