1 Background
After training the network model using Pytorch, the open source deep learning framework, it is usually necessary to convert the format before deployment, for example, the Horizon Toolchain model conversion currently only supports Caffe1.0 and ONNX(opset_version=10/11 and ir_version≤7). ONNX(Open Neural Network Exchange) format is a common open source neural network format, supported by many inference engines, such as Pytorch, PaddlePaddle, TensorFlow, etc. This article details how to export a model in Pytorch format to a model in ONNX format.
2 Experimental environment
This document uses Python3.6 as an example. The whl package and version information are as follows:
torch 1.10.2 onnx 1.8.0 onnxruntime 1.10.0 numpy 1.19.5
3 torch.onnx.export function introduction
The torch.onnx.export function implements the export of the Pytorch model to the ONNX model. In pytorch1.10.2, torch.onnx.export function parameters are as follows:
def export(model, args, f, export_params=True, verbose=False, training=TrainingMode.EVAL,
input_names=None, output_names=None, operator_export_type=None,
opset_version=None, _retain_param_name=None, do_constant_folding=True,
example_outputs=None, strip_doc_string=None, dynamic_axes=None,
keep_initializers_as_inputs=None, custom_opsets=None, enable_onnx_checker=None,
use_external_data_format=None):
You can use the default Settings for most parameters. The following describes some commonly used parameters:
torch.onnx.export(
model,
args,
f,
input_names=None,
output_names=None,
opset_version=11
)
The introduction of other parameters refer to [the official torch. Onnx. Export () function manual] (https://pytorch.org/docs/1.10/_modules/torch/onnx.html#export).
4 Single input network export ONNX model code operation
This section mainly includes three parts: single-input network construction, model export generating ONNX format, and exported ONNX model validity verification. You can directly run the following code to get the corresponding ONNX model, please refer to the code comments for understanding.
import torch.nn as nn
import torch
import numpy as np
import onnx
import onnxruntime
class MyNet(nn.Module):
def __init__(self, num_classes=10):
super(MyNet, self).__init__()
self.features = nn.Sequential(
nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1), # input[3, 28, 28] output[32, 28, 28]
nn.BatchNorm2d(32),
nn.ReLU(inplace=True),
nn.Conv2d(32, 64, kernel_size=3, stride=2, padding=1), # output[64, 14, 14]
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=2) # output[64, 7, 7]
)
self.fc = nn.Linear(64 * 7 * 7, num_classes)
def forward(self, x):
x = self.features(x)
x = torch.flatten(x, start_dim=1)
x = self.fc(x)
return x
def model_convert_onnx(model, input_shape, output_path):
dummy_input = torch.randn(1, 3, input_shape[0], input_shape[1])
input_names = ["input1"]
output_names = ["output1"]
torch.onnx.export(
model,
dummy_input,
output_path,
verbose=False,
opset_version=11,
input_names=input_names,
output_names=output_names,
)
if __name__ == '__main__':
model = MyNet()
model.eval()
input_shape = (28, 28)
output_path = './MyNet.onnx'
model_convert_onnx(model, input_shape, output_path)
print("model convert onnx finsh.")
# import onnxsim
# MyNet_sim = onnxsim.simplify(onnx.load(output_path))
# onnx.save(MyNet_sim[0], "MyNet_sim.onnx")
onnx_model = onnx.load(output_path)
onnx.checker.check_model(onnx_model)
print("onnx model check_1 finsh.")
x = torch.randn(size=(1, 3, input_shape[0], input_shape[1]))
with torch.no_grad():
torch_out = model(x)
print(torch_out) # tensor([[-0.5728, 0.1695, ..., -0.3256, 1.1357, -0.4081]])
# print(type(torch_out)) # <class 'torch.Tensor'>
ort_session = onnxruntime.InferenceSession(output_path)
ort_inputs = {ort_session.get_inputs()[0].name: x.numpy()}
ort_outs = ort_session.run(None, ort_inputs)
# print(ort_outs) # [array([[-0.5727689 , 0.16947027, ..., -0.32555276, 1.13574252, -0.40812433]], dtype=float32)]
# print(type(ort_outs))
# print(type(ort_outs[0]))
ort_outs = ort_outs[0]
# print(torch_out.numpy().shape) # (1, 10)
# print(ort_outs.shape) # (1, 10)
np.testing.assert_allclose(torch_out.numpy(), ort_outs, rtol=1e-03, atol=1e-05)
print("onnx model check_2 finsh.")
5 Multi-input network export ONNX model code operation
This section mainly includes three parts: multi-input network construction, model export generating ONNX format, and exported ONNX model validity verification. You can directly run the following code to get the corresponding ONNX model, please refer to the code comments for understanding.
import torch.nn as nn
import torch
import numpy as np
import onnx
import onnxruntime
class MyNet_multi_input(nn.Module):
def __init__(self, num_classes=10):
super(MyNet_multi_input, self).__init__()
self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=2, padding=1) # input[3, 28, 28] output[32, 14, 14]
self.bn1 = nn.BatchNorm2d(32)
self.relu1 = nn.ReLU(inplace=True)
self.conv2 = nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1) # input[1, 28, 28] output[32, 14, 14]
self.bn2 = nn.BatchNorm2d(16)
self.relu2 = nn.ReLU(inplace=True)
self.fc = nn.Linear(48 * 14 * 14, num_classes)
def forward(self, x, y):
x = self.relu1(self.bn1(self.conv1(x)))
y = self.relu2(self.bn2(self.conv2(y)))
z = torch.cat((x, y), 1)
z = torch.flatten(z, start_dim=1)
z = self.fc(z)
return z
def multi_input_model_convert_onnx(model, input_shape, output_path):
dummy_input1 = torch.randn(1, 3, input_shape[0], input_shape[1])
dummy_input2 = torch.randn(1, 1, input_shape[0], input_shape[1])
input_names = ["input1", "input2"]
output_names = ["output1"]
torch.onnx.export(
model,
(dummy_input1, dummy_input2),
output_path,
verbose=False,
opset_version=11,
input_names=input_names,
output_names=output_names,
)
if __name__ == '__main__':
multi_input_model = MyNet_multi_input()
# print(multi_input_model)
multi_input_model.eval()
input_shape = (28, 28)
multi_input_model_output_path = './multi_input_model.onnx'
multi_input_model_convert_onnx(multi_input_model, input_shape, multi_input_model_output_path)
print("multi_input_model convert onnx finsh.")
# import onnxsim
# multi_input_model_sim = onnxsim.simplify(onnx.load(multi_input_model_output_path))
# onnx.save(multi_input_model_sim[0], "multi_input_model_sim.onnx")
onnx_model = onnx.load(multi_input_model_output_path)
onnx.checker.check_model(multi_input_model_output_path)
print("onnx model check_1 finsh.")
x = torch.randn(size=(1, 3, input_shape[0], input_shape[1]))
y = torch.randn(size=(1, 1, input_shape[0], input_shape[1]))
with torch.no_grad():
torch_out = multi_input_model(x, y)
# print(torch_out) # tensor([[-0.5728, 0.1695, ..., -0.3256, 1.1357, -0.4081]])
# print(type(torch_out)) # <class 'torch.Tensor'>
ort_session = onnxruntime.InferenceSession(multi_input_model_output_path)
ort_inputs = {ort_session.get_inputs()[0].name: x.numpy(), ort_session.get_inputs()[1].name: y.numpy()}
ort_outs = ort_session.run(None, ort_inputs)
# print(ort_outs)
# print(type(ort_outs))
# print(type(ort_outs[0]))
ort_outs = ort_outs[0]
# print(torch_out.numpy().shape) # (1, 10)
# print(ort_outs.shape) # (1, 10)
np.testing.assert_allclose(torch_out.numpy(), ort_outs, rtol=1e-03, atol=1e-05)
print("onnx model check_2 finsh.")
(https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html)。。)
6 ONNX model visualization
Once exported into the ONNX model, you can use the open source visualization tool Netron to view the network structure and related configuration information. Netron use mainly divided into two kinds, one kind is to use online web (https://netron.app/), the other is a download/installation program (https://github.com/lutzroeder/netron). The following is an example of exporting the single input ONNX model in Section 4 of the online web version opening. Click online web version link to open the exported ONNX model. The visualization effect is as follows:
image.png
The ONNX model supported by the Horizon Toolchain needs to be met opset_version=10/11 且 ir_version≤7。
7 ir_version and opset_version changes
The ONNX model supported by the Horizon Toolchain needs to meet opset_version=10/11 and ir_version≤7. What if the ONNX model does not meet these two requirements? This is a solution if there are conditions to modify the code and re-export it. Another solution to try is to directly modify the corresponding properties of the ONNX model, with the following code example:
import onnx
model = onnx.load("./MyNet.onnx")
model.ir_version = 6
model.opset_import[0].version = 10
onnx.save_model(model, "MyNetOutput.onnx")
** Note ** : There may be problems when switching from the high version to the low version, here is only one solution to try. Visualize MyNetoutput.onnx with Netron, as shown below:
image.png