[Rising Sun x3] yolov5seg instance splits cpp deployment code
1. Preface
yolov5-7.0 continues to update the code of instance segmentation, and its accuracy and speed of segmentation are surprising. Python code can be seen in the following post. In this paper, the yolov5-seg model is run on x3, and partial post-processing is carried out by matrix operation method. Take advantage of how to extract the segmentation and detection results from the cpp code.
yolov5seg.png
https://github.com/Rex-LK/tensorrt_learning/tree/main/trt_cpp/src/trt/demo-infer/yolov5seg
https://github.com/shouxieai/algorithm-cpp
https://github.com/Rex-LK/ai_arm_learning/tree/master/x3/x3_inference
2. Instance segmentation results
The result of yolov5-seg is divided into two parts. One is the result of detection, the dimension is 25200*177, the first 85 is listed as the result of each detection box, and the last 32 is listed as the mask coefficient of each detection box. The other is the segmentation result: prototype mask, the dimension is 32 * 160 * 160. The post-processing of instance segmentation is to sum the mask coefficient in the target box and the prototype mask weighted, so as to obtain the effect of instance segmentation. 
yolact.png 以下为部分cpp代码
float *seg_det = seg_out->cpu<float>();
vector<float> mask(seg_det, seg_det + segChannels * segWidth * segHeight);
Matrix seg_proto(segChannels, segWidth * segHeight, mask);
for (int i = 0; i < box_result.size(); ++i) {
Matrix resSeg = (mygemm(box_result[i].mask_cofs,seg_proto).exp(-1) + 1.0).power(-1);
Mat resMat(resSeg.data_);
resMat = resMat.reshape(0,{segHeight,segWidth});
resize(resMat, resMat, Size(INPUT_H,INPUT_W), INTER_NEAREST);
Rect temp_rect = box_result[i].box;
Mat binaryMat;
inRange(resMat(temp_rect), 0.5, 1, binaryMat);
box_result[i].boxMask = binaryMat;
// imwrite(to_string(i) + "_.jpg", b);
}
The following matrices are implemented using cpp, and some simple matrix operations can be implemented.
class Matrix{
public:
Matrix();
Matrix(int rows, int cols, const std::initializer_list<float>& pdata={});
Matrix(int rows, int cols, const std::vector<float>&v);
const float& operator()(int irow, int icol)const {return data_[irow * cols_ + icol];}
float& operator()(int irow, int icol){return data_[irow * cols_ + icol];}
Matrix element_wise(const std::function<float(float)> &func) const;
Matrix operator*(const Matrix &value) const;
Matrix operator*(float value) const;
Matrix operator+(float value) const;
Matrix operator-(float value) const;
Matrix operator/(float value) const;
int rows() const{return rows_;}
int cols() const{return cols_;}
Matrix view(int rows, int cols) const;
Matrix power(float y) const;
float reduce_sum() const;
float* ptr() const{return (float*)data_.data();}
Matrix exp(float value);
public:
int rows_ = 0;
int cols_ = 0;
std::vector<float> data_;
};
3. Test
Download this code
cd x3_inference
mkair build && cd build
cmake .. && make -j
./seg_demo
output-seg.jpg
4. Summary
This time, I have learned the principle and code of yolov5 instance segmentation, and through the comparison principle and code steps, I have found out how yolov5 realizes the task of instance segmentation. If this article is useful to you, please point a star and understand the result processing when there are multiple outputs on the bpu.