PTQ Configuration File Explained

D-Robotics · September 1, 2023, 2:57am

1 Parameter summary table

2 Parameter description

2.1 Model parameter group

‘prototxt’, ‘caffe_model’, ‘onnx_model’ are used to configure the model file. The caffe1.0 model is configured with only prototxt and caffe_model; Only onnx_model is configured for the ONNX model. For example, ONNX model configuration:

onnx_model: ***.onnx
`

‘march’ is a mandatory parameter that specifies the platform architecture to be deployed for the compiled.bin model. The J5 platform must be configured as bayes and the XJ3 platform must be configured as bernoulli2. For example, the J5 platform model is configured as:

march: bayes
` ` `

3. 'output_model_file_prefix' is an optional parameter used to configure the name prefix of the output model file. The default value is model. For example, if mobilenetv1_224x224_nv12 is configured, all ONNX and bin model files produced by the tool are prefixed with this:
```Yaml
output_model_file_prefix: mobilenetv1_224x224_nv12
` ` `
! [](/api/v1/static/imgData/1693536117734.png)

4. 'working_dir' is an optional parameter to configure the path where the output is stored. The default value is model_output. If the directory does not currently exist, the tool automatically creates it.
```Yaml
working_dir: 'model_output'
` ` `



6. 'output_nodes' is an optional parameter that specifies output nodes for the model. Using the '03_classification/01_mobilenet' classification model as an example, if the following configuration is configured, the two output nodes are added starting from the _original_float.onnx model.
```Yaml
output_nodes: 'conv5_6/sep;  conv6/dw' # Simply configure the node name
` ` `
! [](/api/v1/static/imgData/1693536426683.png)

7. 'remove_node_type' and 'remove_node_name' are optional parameters without default values. An operator or operators used to connect the head/tail of the model to the input/output is removed from the model. Currently, the node types supported for deletion include Quantize, Dequantize, Cast, Transpose, and Softmax. The Quantize and Dequantize nodes cannot view the node name from the conversion log. You need to use the hb_perf tool (J5 [link] (HTTP: / / https://developer.horizon.cc/api/v1/fileData/horizon_j5_open_explorer_cn_doc/oe_mapper/source/ptq/ptq_tool/hb_perf. html); XJ3  [link] (HTTP: / / https://developer.horizon.cc/api/v1/fileData/horizon_xj3_open_explorer_cn_doc/oe_mapper/source/ptq/ptq_tool/hb_perf .html)) Visual bin model viewing. Two configurations are provided below (** note the node order **), both of which remove the Softmax and Dequantize nodes in the tail of the mobilenetv1 model.


```Yaml
remove_node_type: "Softmax; Dequantize"    # or
remove_node_name: "prob; fc7_1_HzDequantize"

After the deletion is successful, you can view the following information in the conversion log:

Note that the Quantize and Dequantize nodes are integrated into the model by default using CPU calculations, but it is more efficient to remove them and incorporate them into the post-processing code to avoid redundant data traversal operations.

debug_mode This parameter is optional and has no default value.

debug_mode: dump_calibration_data

2.2 Enter the information parameter group

This section uses the ‘05_miscellaneous/04_multi_input_example’ multi-input model as an example

‘input_name’ is used to specify the name of the model input node. When the original model has only one input node, it can not be configured, and the tool will analyze it automatically. This parameter must be configured for multiple inputs, as shown in the following example:

input_name: image_b; image_g; image_r
`
1. ‘input_shape’ is used to specify the data size of the model input node. When the model input is static size, it is optional, and the tool automatically analyzes; If the model input is dynamic size, it must be configured. And if the model has multiple inputs, it should also be in the same order as’ input_name '. In this example, the size is static and can be configured as follows:
```
input_shape: 1x1x224x224; 1x1x224x224; 1x1x224x224
```

‘input_type_train’ and ‘input_layout_train’ are mandatory parameters, which are used to configure the input data type and arrangement during model training. If the model has multiple inputs, it should also be in the same order as’ input_name '. The preceding example can be configured as follows:

input_type_train: gray; gray; gray
input_layout_train: NCHW; NCHW; NCHW
`

input_type_rt and input_layout_rt are used to configure the type and arrangement of runtime data during.bin model deployment. The former is mandatory, and the latter is optional. If the model has multiple inputs, it should also be in the same order as’ input_name '. The preceding example can be configured as follows:

input_type_rt: gray;  gray;  gray
input_layout_rt: NCHW;  NCHW;  NCHW # can also be left unconfigured
` ` `

5. input_space_and_range is an optional parameter used to configure the data format for input_type_rt: nv12. If the value is regular, the data range is [0, 255]. If bt601_video is configured, the data range is somewhat compressed and is [16, 235]. The default value of this parameter is regular. This parameter is usually not required.
```Yaml
# input_space_and_range: # Usually not configured
` ` `

6. 'input_batch' is optional and the default value is 1. When the model is single input and the first dimension of 'input_shape' is 1, the model can be compiled into batch model using this parameter, which currently supports the configuration range [1-4096]. For example, if you want to compile a single input model of batch4, you can configure the following, ** More information can be found in Section 4.4 of this article **.
```Yaml
input_batch: 4
` ` `
Note: The multi-input model does not yet support the configuration of the input_batch parameter, but the multi-input model of batch supports the configuration of the input_shape parameter to describe the batch size.

## 2.3 Calibration parameter group
1. cal_data_dir and cal_data_type are used to specify the path and type of the calibration data store. Among them:
- If the calibration_type calibration mode is load or skip, neither parameter can be configured.
- If the calibration data is JPEG original and the following 'preprocess_on' is set to True, then 'cal_data_type' may not be set;
- If the calibration data is preprocessed and saved as np.tofile (see more: [image data preparation of calibration problem is introduced and processing] (https://developer.horizon.cc/forumDetail/118363912788935513)), cal_data_type tells the tool whether it should be read in float32 or uint8; If not configured, and in non-LOAD /skip calibration mode, the tool will resolve whether the suffix of the storage folder name is **_f32** to determine the data read type. When the cal_data_type configuration conflicts with the suffix name, it has a higher priority.
- If the model has multiple inputs, the two parameters must be in the same order as' input_name '. The configuration of the multi-input model in the preceding example is as follows:


```Yaml
cal_data_dir: ./calibration_data_b; ./calibration_data_g; ./calibration_data_r
cal_data_type: uint8; uint8; uint8

preprocess_on Is optional parameter, the default value is False, ** and only applies to the model of 4-dimensional image input **, indicating whether the tool automatically processes the image, including using skimage to read the image, and resize to the size of the input node, the user can choose to use according to the situation, usually False.

preprocess_on: False

Parameter name

Parameter Description

Optional/required

Default value

Support platform

model parameter set

prototxt

Specifies the name of the prototxt file for the Caffe floating point model

< td rowspan = “3” > < font color = “red” > two alternative models

/

XJ3, J5

caffe_model

Specifies the caffemodel file name for the Caffe floating point model

/

XJ3, J5

onnx_model

Specifies the ONNX file name for the onnx floating point model

/

XJ3, J5

march

Specifies the platform architecture that needs to be supported to produce hybrid heterogeneous models

Mandatory

/

XJ3, J5

output_model_file_prefix

Specifies the name prefix

for the hybrid heterogeneous model produced by the transformation

Optional

model

XJ3, J5

working_dir

Specifies where to store the results of the model transformation output

Optional

model_output

XJ3, J5

layer_out_dump

Specifies whether the hybrid heterogeneous model retains the ability to output middle-tier values

Optional

False

XJ3, J5

output_nodes

Specifies the output node of the model

Optional

/

XJ3, J5

remove_node_type

Set the type of node to be deleted

Optional

/

XJ3, J5

remove_node_name

Set the name of the node to be deleted

Optional

/

XJ3, J5

debug_mode

Save calibration data for precision debug analysis

Optional

/

XJ3, J5

set_node_data_type

Specifies the output of the operator int16, whose functions are integrated into node_info

Optional

/

J5

node_info

Specifies the operator int16 input/output, or forces the operator to run on the CPU or BPU

Optional

/

J5

< th colspan = “5” > < font size = “3” color = “blue” > input parameter set < / font >

input_name

Specifies the input node name for the original floating point model

Optional

/

XJ3, J5

input_type_train

Specifies the input data type of the original floating point model

Mandatory

/

XJ3, J5

input_layout_train

Specifies the input data layout of the original floating point model

Mandatory

/

XJ3, J5

input_type_rt

Input data format to be adapted to the converted mixed heterogeneous model

Mandatory

/

XJ3, J5

input_layout_rt

Input data configuration required for the converted hybrid heterogeneous model

Optional

/

XJ3, J5

input_space_and_range

Special format for specifying the input data format

Optional

regular

XJ3, J5

input_shape

Specifies the input data size of the original floating point model

Optional

/

XJ3, J5

input_batch

Specifies the number of input batches to be adapted to the converted hybrid heterogeneous model

Optional

1

XJ3, J5

norm_type

Input data preprocessing methods added to the model

Optional

no_preprocess

XJ3, J5

mean_value

Specifies the mean of the image subtracted from the preprocessing method

Optional

/

XJ3, J5

scale_value

Specifies the numerical scale coefficient of the preprocessing method

Optional

/

XJ3, J5

Calibration parameter set (All parameters are not independently optional, can not be configured)

cal_data_dir

Specify where to store calibration samples for model calibration

Optional

/

XJ3, J5

cal_data_type

Specify the data storage type for the calibration data binary

Optional

/

PTQ Configuration File Explained

Directory

1 Parameter summary table

2 Parameter description

2.1 Model parameter group

2.2 Enter the information parameter group