基于实例分割的休眠期枣树修剪枝参数提取方法

doi:10.13733/j.jcam.issn.2095-5553.2024.10.008

摘要/Abstract

摘要： 为实现休眠期枣树选择性修剪，针对自动化剪枝过程中枣树枝干识别与参数提取困难的问题，提出一种基于实例分割神经网络的枝干识别与参数自动提取的方法。首先，通过前期搭建的视觉系统获取2个角度下的点云，并基于骨架点重建完整的枣树点云，利用CloudCompare V 2.11软件对枣树三维点云进行手工标注，构建带语义信息的枣树点云数据集，将标注完的376棵枣树，按照8∶2的比例分为训练集和验证集；其次，结合JSNet实例分割网络对枣树点云进行分割，同时对比分析不同自然环境对实例分割精度的影响；最后，提出枣树修剪枝直径和长度参数自动提取的方法。试验结果表明：休眠期枣树主干的分割精度为96%，而修剪枝的分割精度为77%，并且不同自然环境对分割精度影响较小。枣树修剪枝的直径拟合值与实际测量值误差范围在2 mm以内，且其长度拟合值与实际值的误差范围在1 cm以内，为后续准确确定枣树剪枝点的位置提供数据依据。

关键词: 休眠期枣树, 实例分割, 深度学习, 三维点云, 修剪枝

Abstract: In order to achieve selective pruning of dormant jujube tree, the identification and parameter extraction method of pruning branches of dormant jujube tree was proposed based on instance segmentation neural network, in view of the difficulty of extracting pruning parameters in automatic pruning process. Firstly, point cloud from two angles were obtained through the vision system built earlier, and a complete point cloud of date tree was reconstructed based on skeleton points. The three‑dimensional point cloud of date tree was manually labeled by CloudCompare V 2.11 software, and dataset of date tree with semantic information was constructed. The labeled 376 trees were divided into training and validation dataset according to the ratio of 8∶2, and then instance segmentation of dormant jujube tree was carried out by JSNet neural network. The influence of different natural environment for the segmentation accuracy was analyzed. Secondly, the fitting calculation method for diameter and length parameters of pruning branches was proposed. The experimental results showed that the segmentation accuracy of jujube tree trunk was 96%, and that of pruning branches was 77%, which was less affected by the natural environment. The relative error between the diameter fitting result and the actual value was less than 2 mm, and the error between the length value and actual measurement value was less than 1 cm, which provided a basis for the accurate determination of the location of pruning point of jujube tree.

Key words: dormant jujube tree, instance segmentation, deep learning, three?dimensional point cloud, pruning branches

中图分类号:

马保建, 陈棒棒, 李学志, 蒋焕煜. 基于实例分割的休眠期枣树修剪枝参数提取方法[J]. 中国农机化学报, 2024, 45(10): 54-59.

Ma Baojian, Chen Bangbang, Li Xuezhi, Jiang Huanyu. Extraction method of pruning parameters of dormant jujube tree based on instance segmentation[J]. Journal of Chinese Agricultural Mechanization, 2024, 45(10): 54-59.

参考文献

[ 1 ] 付昱兴, 李承明, 朱江, 等. Alpha‑shape算法构建枣树点云三维模型[J]. 农业工程学报, 2020, 36(22): 214-221.
Fu Yuxing, Li Chengming, Zhu Jiang, et al. Three‑dimensional model construction method and experiment of jujube tree point cloud using Alpha‑shape algorithm [J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(22): 214-221.
[ 2 ] 郑永军, 江世界, 陈炳太, 等. 丘陵山区果园机械化技术与装备研究进展[J]. 农业机械学报, 2020, 51(11): 1-20.
Zheng Yongjun, Jiang Shijie, Chen Bingtai, et al. Review on technology and equipment of mechanization in hilly orchard [J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(11): 1-20.
[ 3 ] Zahid A, He L, Zeng L, et al. Development of a robotic end‑effector for apple tree pruning [J]. Transactions of the ASABE, 2020, 63(4): 847-856.
[ 4 ] Tabb A, Medeiros H. Automatic segmentation of trees in dynamic outdoor environments [J]. Computers in Industry, 2018, 98: 90-99.
[ 5 ] 马保建, 鄢金山, 王乐, 等. 基于语义分割的矮化密植枣树修剪枝识别与骨架提取[J]. 农业机械学报, 2022, 53(8): 313-319.
Ma Baojian, Yan Jinshan, Wang Le, et al. Method for detection and skeleton of pruning branch of jujube tree based on semantic segmentation for dormant pruning [J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(8): 313-319.
[ 6 ] Fourie J, Bateman C, Hsiao J, et al. Towards automated grape vine pruning: Learning by example using recurrent graph neural networks [J]. International Journal of Intelligent Systems, 2021, 36(2): 715-735.
[ 7 ] Fernandes M, Scaldaferri A, Fiameni G, et al. Grapevine winter pruning automation: On potential pruning points detection through 2D plant modeling using grapevine segmentation [J]. arxiv preprint arxiv: 2106. 04208, 2021.
[ 8 ] You A, Parayil N, Krishna J G, et al. An autonomous robot for pruning modern, planar fruit trees [J]. arxiv preprint arxiv: 2206. 07201, 2022.
[ 9 ] Verbiest R, Ruysen K, Vanwalleghem T, et al. Automation and robotics in the cultivation of pome fruit: Where do we stand today? [J]. Journal of Field Robotics, 2021, 38(4): 513-531.
[10] Karkee M, Adhikari B, Amatya S, et al. Identification of pruning branches in tall spindle apple trees for automated pruning [J]. Computers and Electronics in Agriculture, 2014, 103: 127-135.
[11] Elfiky N M, Akbar S A, Sun J, et al. Automation of dormant pruning in specialty crop production: An adaptive framework for automatic reconstruction and modeling of apple trees [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015: 65-73.
[12] You A, Grimm C, Silwal A, et al. Semantics‑guided skeletonization of sweet cherry trees for robotic pruning [J]. arxiv preprint arxiv: 2103. 02833, 2021.
[13] Fu Y, Xia Y, Zhang H, et al. Skeleton extraction and pruning point identification of jujube tree for dormant pruning using space colonization algorithm [J]. Frontiers in Plant Science, 2022, 13.
[14] Ma B, Du J, Wang L, Jiang H, Zhou M. Automatic branch detection of jujube trees based on 3D reconstruction for dormant pruning using the deep learning‑based method [J]. Computers and Electronics in Agriculture. 2021, 190.
[15] Zhao L, Tao W. JSNet: Joint instance and semantic segmentation of 3D point clouds [C]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12951-12958.
[16] Qi C R, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space [J]. Advances in Neural Information Processing Systems, 2017, 30.
[17] Wu W, Qi Z, Fuxin L. Pointconv: Deep convolutional networks on 3D point clouds [C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 9621-9630.
[18] Comaniciu D, Meer P. Mean shift: A robust approach toward feature space analysis [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 603-619.

[1]	韩鹏飞, 宋其江, 贾梦实. 基于改进轻量化 EfficientNet-V2模型的小麦种子分类[J]. 中国农机化学报, 2024, 45(9): 111-117.
[2]	李志臣, 凌秀军, 李鸿秋. 基于声音深度学习的发动机失火故障诊断[J]. 中国农机化学报, 2024, 45(9): 134-140.
[3]	桑一男 , 徐增莱 , 汪琼 , 葛海涛 , 王殿广 . 一种轮式采花机器人设计——以黄蜀葵为例[J]. 中国农机化学报, 2024, 45(9): 202-208.
[4]	付晓鸽, 李涵, 左治江, 杜铮 . 基于 Mask R-CNN的复杂环境下辣椒识别方法研究[J]. 中国农机化学报, 2024, 45(9): 215-219.
[5]	李邦国, 王辉, 宋杨, 任志伟, 刘跃华, 徐乐程. 基于无人驾驶小麦收割机立体视觉感知系统[J]. 中国农机化学报, 2024, 45(9): 244-249.
[6]	许贞辉, 李晓娟. 果园树干检测与导航线拟合算法研究[J]. 中国农机化学报, 2024, 45(8): 217-222.
[7]	胡国玉, 刘广, 周星光, 董娅兰, 周建平, . 基于Swin-TDL算法的果园环境下葡萄病害检测方法[J]. 中国农机化学报, 2024, 45(8): 234-239.
[8]	吴利刚, 吕媛媛, 周倩, 陈乐, 张梁, 史建华. 基于改进YOLOv5的轻量级黄花成熟检测方法[J]. 中国农机化学报, 2024, 45(7): 235-242.
[9]	张继成, 黄向党. 基于改进YOLOv3的玉米病害识别方法[J]. 中国农机化学报, 2024, 45(7): 269-275.
[10]	雷浩, 苑迎春, 何振学. 基于多尺度卷积和注意力机制的枣品种识别[J]. 中国农机化学报, 2024, 45(6): 135-141.
[11]	张煜恒, 周宏平. 基于激光点云的树木特征信息提取研究进展[J]. 中国农机化学报, 2024, 45(6): 142-148.
[12]	崔海朋, 秦朝旭, 马志宇. 基于深度学习的鱼类特征点检测与体征识别方法[J]. 中国农机化学报, 2024, 45(6): 201-207.
[13]	黄尧, 何敬, 付饶, 刘刚, , 林远杨. YOLOv5s-CBAM算法在福寿螺虫卵识别中的应用分析[J]. 中国农机化学报, 2024, 45(6): 223-228.
[14]	陶旭, , 余富强, , 蔡金金, 么炜, , 刘博, . 面向激光雷达点云数据的多结构树种识别[J]. 中国农机化学报, 2024, 45(5): 168-175.
[15]	漆海霞, , , 李承杰, 黄桂珍. 基于轻量化YOLOv4的死淘鸡目标检测算法[J]. 中国农机化学报, 2024, 45(5): 195-201.