1 Background
After training a network model using a deep learning open source framework, a format conversion is usually required before deployment, and the Horizon Toolchain model conversion currently supports Caffe1.0 and ONNX(opset_version=10/11 and ir_version≤7). ONNX(Open Neural Network Exchange) format is a common open source neural network format, supported by many inference engines, such as Pytorch, PaddlePaddle, TensorFlow, etc. This article details how to export a PaddlePaddle format model to ONNX format.
2 Experimental environment
The experimental environment for this tutorial is as follows:
The Python library
Version
paddlepaddle
2.4.1
paddle2onnx
1.0.5
onnx
1.13.0
onnxruntime
1.14.0
3 Introduction to paddle.onnx.export function
The PaddlePaddle model can be exported to an onnx model by the Paddlepaddle. The functions are described as follows: x_spec is used to configure the input_spec parameter of paddle.onnx.export.
x_spec = paddle.static.InputSpec(shape=None, dtype='float32', name=None)
#shape: Declares dimension information, defaults to None
#dtype: data type. The default is float32
#name: Network input node name
paddle.onnx.export(layer, path, input_spec=[x_spec], opset_version=11, **configs)
#layer: The exported Layer object, that is, the network model to be transformed
#path: Store the path prefix of the model. After exporting, the suffix ".onnx "is automatically added.
#input_spec: Used to configure model input properties
#opset_version: The default value is 9, please manually set 10 or 11
For more details on paddle.onnx.export, see the PaddlePaddle API documentation: https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/onnx/export_cn.html
Code operation
PaddlePaddle is derived with ONNX model
The following code shows the process of building a simple classification model and saving it in PaddlePaddle and ONNX formats.
import paddle
import paddle.nn as nn
class MyNet(nn.Layer):
def __init__(self, num_classes=10):
super(MyNet, self).__init__()
self.num_classes = num_classes
self.features = nn.Sequential(
nn.Conv2D(in_channels=1, out_channels=2,
kernel_size=3, stride=1, padding=1),
nn.ReLU())
self.linear = nn.Sequential(nn.Linear(98, num_classes))
def forward(self, inputs):
x = self.features(inputs)
x = paddle.flatten(x, 1)
x = self.linear(x)
return x
model = MyNet()
# Ready to enter data
x_spec = paddle.static.InputSpec([1, 1, 7, 7], 'float32', 'input1')
# Save the model in PaddlePaddle format to verify the consistency of inference with the ONNX model
paddle.jit.save(layer=model, path='./pd_model/pdmodel',
input_spec=[x_spec])
# Export the model to ONNX format for saving
paddle.onnx.export(layer=model, path='./model',
ONNX correctness verification
You can verify the correctness of the ONNX model with the following code, which checks the version of the model, the structure of the graph, the nodes, and the input and output. If the output is Check: None, no error information is reported and the model is correctly exported.
import onnx
onnx_model = onnx.load("./model.onnx")
check = onnx.checker.check_model(onnx_model)
print('Check: ', check)
Consistency check of PaddlePaddle and ONNX You can use the following code to check that the derived ONNX model and the original PaddlePaddle model have the same calculations.
import numpy as np
import onnxruntime
import paddle
input1 = np.random.random((1, 1, 7, 7)).astype('float32')
ort_sess = onnxruntime.InferenceSession("./model.onnx")
ort_inputs = {ort_sess.get_inputs()[0].name: input1}
ort_outs = ort_sess.run(None, ort_inputs)
model = paddle.jit.load("./pd_model/pdmodel")
model.eval()
paddle_input = paddle.to_tensor(input1)
paddle_outs = model(paddle_input)
print(ort_outs[0])
print(paddle_outs.numpy())
np.testing.assert_allclose(tf_outs.numpy(), ort_outs[0], rtol=1e-03, atol=1e-05)
print("onnx model check finsh.")
Case of multiple inputs
If your model has multiple inputs, you can save them in PaddlePaddle and ONNX format by referring to the code below. The correctness verification of ONNX and the consistency check of PaddlePaddle and ONNX are no longer described, and can be written as the above code.
import paddle
import paddle.nn as nn
class MyNet(nn.Layer):
def __init__(self, num_classes=10):
super(MyNet, self).__init__()
self.num_classes = num_classes
self.features_1 = nn.Sequential(
nn.Conv2D(in_channels=1, out_channels=2,
kernel_size=3, stride=1, padding=1),
nn.ReLU())
self.features_2 = nn.Sequential(
nn.Conv2D(in_channels=1, out_channels=2,
kernel_size=3, stride=1, padding=1),
nn.ReLU())
self.linear = nn.Sequential(nn.Linear(98, num_classes))
def forward(self, inputs1, inputs2):
x = self.features_1(inputs1)
y = self.features_2(inputs2)
z = paddle.concat((x, y), 1)
z = paddle.flatten(z, 1)
z = self.linear(z)
return z
model = MyNet()
x_spec = paddle.static.InputSpec([1, 1, 7, 7], 'float32', 'input1')
y_spec = paddle.static.InputSpec([1, 1, 7, 7], 'float32', 'input2')
paddle.jit.save(layer=model, path='./pd_model/pdmodel',
input_spec=[x_spec, y_spec])
paddle.onnx.export(layer=model, path='./model',
input_spec=[x_spec, y_spec], opset_version=11)
5 ONNX model visualization
Once exported into the ONNX model, you can use the open source visualization tool Netron to view the network structure and related configuration information. Netron use mainly divided into two kinds, one kind is to use the online web page version of https://netron.app/, another kind is to download the installation program at https://github.com/lutzroeder/netron. The visualizations of the model in this tutorial are:
6 ir_version and opset_version are modified
import onnx
model = onnx.load("./model.onnx")
model.ir_version = 6
model.opset_import[0].version = 10
onnx.save_model(model, "./model_version.onnx")
** Note: ** There may be problems when switching from the high version to the low version, here is only one solution to try. After the adjustment, use Netron to visualize model_version.onnx as shown below: At this time, the ir_version=6 and opset_version=10 of the ONNX model meet the conversion conditions of the Horizon tool chain.