Introduction
Docker is an open source application container engine that allows developers to package their applications and dependencies into a portable image and then publish them on any popular Linux or Windows operating system, isolated from each other.
To help you get started quickly with the chip algorithm toolchain, Horizon provides a Docker image containing the full development environment, which greatly simplifies the deployment process of the development environment while avoiding conflicts with the local environment. You can complete the Docker installation according to the first chapter of the following article, and then download the offline image of the tool chain through the introduction of the second chapter. After loading it to the local, you can develop it in the container.
1. Docker install
1.1 Ubuntu
1.1.1 CPU Docker
auto install:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
install by hand：https://www.runoob.com/docker/ubuntu-docker-install.html
1.1.2 GPU Docker
You are advised to install docker ce version 19.03 or later. By default, nvidia docker is built-in. You only need to install Nvidia-Container-Toolkit Please refer to the following command to install it. For more detailed instructions, please refer to the NVIDIA official documentation:
curl https://get.docker.com | sh \ && sudo systemctl --now enable docker
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \ && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
sudo docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
1.2 CentOS
1.2.1 CPU Docker
auto install:
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
install by hand：https://www.runoob.com/docker/centos-docker-install.html
1.2.2 GPU Docker
You are advised to install docker ce version 19.03 or later. By default, nvidia docker is built-in. You only need to install Nvidia-Container-Toolkit Please refer to the following command to install CentOS 7. For more detailed instructions and installation instructions, please refer to the NVIDIA official documentation:
sudo dnf install -y tar bzip2 make automake gcc gcc-c++ vim pciutils elfutils-libelf-devel libglvnd-devel iptables
sudo yum-config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo yum repolist -v
sudo yum install -y https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.4.3-3.1.el7.x86_64.rpm
sudo yum install docker-ce -y
sudo systemctl --now enable docker
sudo docker run --rm hello-world
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \ && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo yum clean expire-cache
sudo yum install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
sudo docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
1.3 Windows
It is not recommended to build an environment of chip algorithm toolchain on a personal laptop, and it is recommended to use a development environment with a memory of at least 16G, otherwise it is easy to have insufficient memory problems when converting and compiling larger models (radar, bev, etc.).
1.3.1 CPU Docker
1. install Docker Desktop： Install Docker Desktop on Windows
** 2.install wsl: ** Start dockerdesktop, may prompt wsl version is too low related problems, the problem will be directly given the official installation link, you can click the link directly follow the steps to complete wsl update. The following steps are for reference only: Open PowerShell as an administrator (Start menu > PowerShell > Right-click > Run as Administrator) and enter the following command to enable the Windows subsystem for Linux:
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
② Run the following command in PowerShell as an administrator to enable the virtual machine platform features:
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
③ Download the Linux kernel update package, and double-click to install: [] X64 download link (https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi) [] arm64 download link (https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_arm64.msi); (You can use the ‘systeminfo’ command to see the local system type) ④ Run the following command in PowerShell as an administrator to set WSL 2 as the default version
wsl --set-default-version 2
**3. Verify that the installation is successful ** Generally speaking, docker will start automatically. If it does not start, you can search for Docker Desktop and double-click to start it. After docker starts normally, you can see a small whale in the notification bar. At this point, you can launch the command window (shortcut key: WIN+R), run the following command to see Hello from Docker! If the message is displayed, the installation is successful.
docker run hello-world
\### 1.3.2 GPU Docker \*\*1. Install WSL2 (only WSL2 can call the GPU) : \*\* Run the following command in PowerShell as an administrator to enable the virtual machine platform features: \`\`\`language dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart \`\`\` ② Download the Linux kernel update package, and double-click to install: \[\] X64 download link (https://wslstorestorage.blob.core.windows.net/wslblob/wsl\_update\_x64.msi) \[\] arm64 download link (https://wslstorestorage.blob.core.windows.net/wslblob/wsl\_update\_arm64.msi); (You can use the 'systeminfo' command to see the local system type) Run the following command in PowerShell as an administrator to set WSL 2 as the default version \`\`\`language wsl --set-default-version 2 \`\`\`
**2. Install the Linux subsystem: ** ① Go to the App Store and install Ubuntu 20.04 LTS ② Start ubuntu and set username and password ③ After the installation is complete, you can run the windows command wsl --list --verbose to view the wsl version of Ubuntu:
**3. Install the gpu basic running environment ** You also need to install cuda in the system. After cuda is installed correctly, you can refer to Section 1.1 above to install nvidia-docker on the ubuntu system
2. Horizon chip Algorithm Tool Chain Docker environment deployment
2.1 Downloading an offline image
Download link for the latest Toolchain development environment image from the Horizon Developer community: X3 Pai: You can refer to the user manual to complete the quick model conversion environment installation without installing docker: https://developer.horizon.ai/api/v1/fileData/documents_pi/ai_toolchain_develop/env_install.html XJ3 tool chain development kit and mirror download: https://developer.horizon.ai/forumDetail/136488103547258769 J5 tool chain development kit and mirror download: https://developer.horizon.ai/forumDetail/118363912788935318
2.2 Loading the offline image and starting the container
After installing the docker basic environment according to Chapter 1, use the following commands to load the offline image and run the container: **1. Load image **
docker load -i docker_openexplorer_xxx.tar.gz
After loading the image, you can view all the local images by the command ‘docker images’ :
**2. Run the container ** ① If you have downloaded the OpenExplorer development package, decompress it, go to the root directory of the package, and run the script to enter the container ```language sh run_docker.sh /data ``` You can also enter a command to enter the container CPU docker ```language docker run -it --rm -v `pwd`:/open_explorer {REPOSITORY}:{TAG} ``` GPU docker ```language docker run -it --rm --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all --shm-size=“15g” -v `pwd`:/workspace {REPOSITORY}:{TAG} ``` **3. Run the following command to verify that you have successfully entered the toolchain development environment **
hb_mapper --help
3. Docker commands
Image : The Docker Image (Image) is equivalent to a root file system. For example, the official image ubuntu:16.04 contains a complete set of Ubuntu16.04 minimal system root file system. Container : The relationship between Image and Container, just like classes and instances in object-oriented programming, the image is the static definition, and the container is the entity in which the image is run. Containers can be created, started, stopped, deleted, paused, and so on. Repository : The repository source for the mirror.
function The
command
docker pull {REPOSITORY}:{Tag}
Loading an offline image
docker load -i image_name.tar
View the local image
docker images
Starts the cpu container
docker run-it --rm -v \ ‘pwd’ :/workspace {REPOSITORY}:{TAG} or: docker run -it -v `pwd`:/workspace {IMAGE ID}
Starts the gpu container
docker run -it --rm --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=compute, utility-e NVIDIA_VISIBLE_DEVICES=all --shm-size=“15g” -v `pwd`:/workspace {REPOSITORY}:{TAG}
Check out the local container
docker ps
Connect to the local CONTAINER
docker exec -it {CONTAINER ID} /bin/bash
stop the CONTAINER
docker stop {CONTAINER ID}
Opening a CONTAINER
docker start {CONTAINER ID}
Deleting a CONTAINER
docker rm -f {CONTAINER ID}
Deleting an IMAGE
docker rmi -f {IMAGE ID}
save your image
1. After installing the environment in the CONTAINER, create an image from the container: docker commit {CONTAINER ID} {Repository}:{Tag} 2. save the image: docker save -o image_name.tar {Repository}:{Tag} 3. To compress the image file, run the docker save {Repository}:{Tag}