基于YOLOv3算法与3D视觉的农业采摘机器人目标识别与定位研究

doi:10.13733/j.jcam.issn.2095-5553.2022.12.026

摘要/Abstract

摘要： 为解决目前农业采摘机器人目标难以识别与定位的问题，在原有农业采摘机器人的基础上，提出一种改进YOLOv3算法和3D视觉技术相结合的方法，实现目标的准确识别和精准定位，并利用标定完成目标坐标系和机器人坐标系的转换。通过试验分析改进YOLOv3算法的性能，并与之前的YOLOv3算法、Fast RCNN算法和Faster RCNN算法进行综合比较，研究表明所采用的改进YOLOv3算法和3D视觉具有较高的识别准确度和定位精度，识别准确率分别提高55%、9%、1.4%，最大定位误差分别降低0.69、0.44、0.28 mm，可以较好地完成后续采摘工作，对于农业机器人的发展具有重要的参考价值。

关键词: YOLOv3, 视觉, 采摘机器人, 识别与定位

Abstract: In order to solve the current agricultural picking robot problem of target recognizing and localizing difficulties, this paper uses an improved YOLOv3 algorithm combined with 3D vision technology to achieve accurate recognition and localization of the target based on the original agricultural picking robot, and completes the transformation between the target coordinate system and the robot coordinate system by calibration. The performance of improved YOLOv3 algorithm is analyzed through experiments, and compared with the previous YOLOv3 algorithm, Fast RCNN algorithm and Faster RCNN algorithm. The research shows that improved YOLOv3 algorithm and 3D vision have higher recognizing accuracy and localizing accuracy, and the recognizing accuracy is increased by 5.5%, 9% and 1.4% respectively, and the maximum localizing errors were reduced by 0.69, 0.44 and 0.28 mm respectively. It can better complete the followup picking work, which has important reference value for the development of agricultural robot.

Key words: YOLOv3, vision, picking robot, recognition and localization

中图分类号:

TP249

高帅, 刘永华, 高菊玲, 吴丹, 姬丽雯, . 基于YOLOv3算法与3D视觉的农业采摘机器人目标识别与定位研究[J]. 中国农机化学报, 2022, 43(12): 178-183.

Gao Shuai, Liu Yonghua, Gao Juling, Wu Dan, Ji Liwen. Research on target recognition and localization of agricultural picking robot based on YOLOv3 algorithm and 3D vision[J]. Journal of Chinese Agricultural Mechanization, 2022, 43(12): 178-183.

参考文献

［1］　
罗锡文, 廖娟, 邹湘军, 等. 信息技术提升农业机械化水平［J］. 农业工程学报, 2016, 32(20): 1-14.

Luo Xiwen, Liao Juan, Zou Xiangjun, et al. Enhancing agricultural mechanization level through information technology ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2016, 32(20): 1-14.

［2］　
何文斌, 魏爱云, 明五一, 等. 基于机器视觉的水果品质检测综述［J］. 计算机工程与应用, 2020, 56(11): 10-16.

He Wenbin, Wei Aiyun, Ming Wuyi, et al. Survey of fruit quality detection based on machine vision ［J］. Computer Engineering and Applications, 2020, 56(11): 10-16.

［3］　
杨涛, 李晓晓. 机器视觉技术在现代农业生产中的研究进展［J］. 中国农机化学报, 2021, 42(3): 171-181.

Yang Tao, Li Xiaoxiao. Research progress of machine vision technology in modern agricultural production ［J］. Journal of Chinese Agricultural Mechanization, 2021, 42(3): 171-181.

［4］　
Zhang L, Jia J D, Gui G. Deep learning based improved classification system for designing tomato harvesting robot ［J］. IEEE Access，2018, 6(9): 67940-67950.

［5］　
Kuznetsova A, Maleva T, V Solovie v. Using YOLOv3 algorithm with pre and postprocessing for apple detection in fruitharvesting robot ［J］. Agronomy, 2020, 10(7): 1016.

［6］　
Ling Xiao, Zhao Yuanshen, Gong Liang, et al. Dualarm cooperation and implementing for robotic harvesting tomato using binocular vision ［J］. Robotics and Autonomous Systems, 2018, 114: 134-143.

［7］　
Ge Y, Xiong Y, Tenorio G L, et al. Fruit localization and environment perception for strawberry harvesting robots ［J］. IEEE Access, 2019, 7: 147642-147652.

［8］　
Feng Qingchun, Zou Wei, Fan Pengfei, et al. Design and test of robotic harvesting system for cherry tomato ［J］. International Journal of Agricultural and Biological Engineering, 2018, 11(1): 96-100.

［9］　
郑太雄, 江明哲, 冯明驰. 基于视觉的采摘机器人目标识别与定位方法研究综述［J］. 仪器仪表学报, 2021, 42(9): 28-51.

Zheng Tai, Jiang Mingzhe, Feng Mingchi. Vision based target recognition and location for picking robot: A review ［J］. Chinese Journal of Scientific Instrument, 2021, 42(9): 28-51.

［10］　
王瑾, 王瑞荣, 李晓红. 番茄采摘机器人目标识别方法研究［J］. 江苏农业科学, 2021, 49(20): 217-222.

［11］　
李寒, 陶涵虓, 崔立昊, 等. 基于SOMKmeans算法的番茄果实识别与定位方法［J］. 农业机械学报, 2021, 52(1): 23-29.

Li Han, Tao Hanxiao, Cui Lihao, et al. Recognition and localization method of tomato based on SOM Kmeans algorithm ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(1): 23-29.

［12］　
穆龙涛, 高宗斌, 崔永杰, 等. 基于改进AlexNet的广域复杂环境下遮挡猕猴桃目标识别［J］. 农业机械学报, 2019, 50(10): 24-34.

Mu Longtao, Gao Zongbin, Cui Yongjie, et al. Kiwifruit detection of farview and occluded fruit based on improved AlexNet ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(10): 24-34.

［13］　
熊俊涛, 刘振, 汤林越, 等. 自然环境下绿色柑橘视觉检测技术研究［J］. 农业机械学报, 2018, 49(4): 45-52.

Xiong Juntao, Liu Zhen, Tang Linyue, et al. Visual Detection Technology of Green Citrus under Natural Environment ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49(4): 45-52.

［14］　
龙洁花, 赵春江, 林森, 等. 改进Mask R-CNN的温室环境下不同成熟度番茄果实分割方法［J］. 农业工程学报, 2021, 37(18): 100-108.

Long Jiehua, Zhao Chunjiang, Lin Sen, et al. Segmentation method of the tomato fruits with different maturities under greenhouse environment based on improved Mask R-CNN ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(18): 100-108.

［15］　
陈新, 伍萍辉, 祖绍颖, 等. 基于改进SSD轻量化神经网络的番茄疏花疏果农事识别方法［J］. 中国瓜菜, 2021, 34(9): 38-44.

Chen Xin, Wu Pinghui, Zu Shaoying, et al. Study on identification method of thinning flower and fruit of tomato based on improved SSD lightweight neural network ［J］. China Cucurbits and Vegetables, 2021, 34(9): 38-44.

［16］　
彭红星, 黄博, 邵园园, 等. 自然环境下多类水果采摘目标识别的通用改进SSD模型［J］. 农业工程学报, 2018, 34(16): 155-162.

Peng Hongxing, Huang Bo, Shao Yuanyuan, et al. General improved SSD model for picking object recognition of multiple fruits in natural environment ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34(16): 155-162.

［17］　
赵辉, 乔艳军, 王红君, 等. 基于改进YOLOv3的果园复杂环境下苹果果实识别［J］. 农业工程学报, 2021, 37(16): 127-135.

Zhao Hui, Qiao Yanjun, Wang Hongjun, et al. Apple fruit recognition in complex orchard environment based on improved YOLOv3 ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(16): 127-135.

［18］　
刘芳, 刘玉坤, 林森, 等. 基于改进型YOLO的复杂环境下番茄果实快速识别方法［J］. 农业机械学报, 2020, 51(6): 229-237.

Liu Fang, Liu Yukun, Lin Sen, et al. Fast Recognition method for tomatoes under complex environments based on improved YOLO ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(6): 229-237.

［19］　
武星, 齐泽宇, 王龙军, 等. 基于轻量化YOLOv3卷积神经网络的苹果检测方法［J］. 农业机械学报, 2020, 51(8): 17-25.

Wu Xing, Qi Zeyu, Wang Longjun, et al. Apple detection method based on lightYOLOv3 convolutional neural network ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(8): 17-25.

［20］　
黄怡蒙, 易阳. 融合深度学习的机器人目标检测与定位［J］. 计算机工程与应用, 2020, 56(24): 181-187.

Huang Yimeng, Yi Yang. Robot object detection and localization based on deep learning ［J］. Computer Engineering and Applications, 2020, 56(24): 181-187.

［21］　
王红君, 牟其松, 岳有军, 等. 基于YOLOv3的水果采摘通用检测模型研究［J］. 中国科技论文, 2021, 16(3): 336-342.

Wang Hongjun, Mou Qisong, Yue Youjun, et al. Research on universal detection model of fruit picking based on YOLOv3 ［J］. China Science Paper, 2021, 16(3): 336-342.

[1]	姜来, 王文娣, 霍晓静, 王辉, 唐娟, 李丽华, (. 死鸡识别机器人系统设计与试验[J]. 中国农机化学报, 2023, 44(8): 81-87.
[2]	刘浩, 孟令峰, 王松峰, 刘自畅, 杜海娜, 孙福山. 基于机器视觉的烤烟鲜烟成熟度判别模型优选[J]. 中国农机化学报, 2023, 44(8): 118-124.
[3]	舒卓, 陈立平, 陈梅香, 张瑞瑞, 郭新宇, 温维亮. 单目多视角图像的美国白蛾三维模型重建系统与试验[J]. 中国农机化学报, 2023, 44(8): 168-173.
[4]	高芳征, 汤文俊, 陈光明, 黄家才. 基于改进YOLOv3的复杂环境下西红柿成熟果实快速识别[J]. 中国农机化学报, 2023, 44(8): 174-183.
[5]	余胜, 谢莉. 基于迁移学习和卷积视觉转换器的农作物病害识别研究[J]. 中国农机化学报, 2023, 44(8): 191-197.
[6]	刘雯雯, 孙裕晶, 姜树辉, 姜鹏. 遮挡情况下的行人检测方法研究[J]. 中国农机化学报, 2023, 44(7): 179-186.
[7]	周红标, 朱亚文, 刘晓洋, 张国良, 李边豪. 基于机器视觉的水稻印刷播种机成种率检测[J]. 中国农机化学报, 2023, 44(7): 194-199.
[8]	王春桃, , , , 梁炜健, 郭庆文, 钟浩, 甘雨, 肖德琴, , . 农业害虫智能视觉检测研究综述[J]. 中国农机化学报, 2023, 44(7): 207-213.
[9]	赵铖钥, 马伟, 苏道毕力格, 谭彧. 激光雷达与视觉融合的跟随运输机器人设计[J]. 中国农机化学报, 2023, 44(6): 176-181.
[10]	罗山, 侯俊涛, 郑彬. 基于机器视觉的石榴品质自动分级方法[J]. 中国农机化学报, 2023, 44(3): 117-122.
[11]	王根, 江晓明, 黄峰, 方迪, 张宇钦. 基于改进YOLOv3网络模型的茶草位置检测算法[J]. 中国农机化学报, 2023, 44(3): 199-207.
[12]	吴俊鹏, 黄光文, 李君, . 水果病害视觉与光谱检测技术研究现状及展望[J]. 中国农机化学报, 2023, 44(2): 112-118.
[13]	李志臣, 凌秀军, 李鸿秋. 基于机器视觉的槟榔分级方法研究[J]. 中国农机化学报, 2023, 44(1): 137-141.
[14]	姬丽雯, , 刘永华, , 高菊玲, , 吴丹, . 温室草莓采摘机器人设计与试验[J]. 中国农机化学报, 2023, 44(1): 192-198.
[15]	梁喜凤, 花瑞, 杨铭涛. 番茄枝叶修剪机械手视觉伺服控制系统研究[J]. 中国农机化学报, 2022, 43(7): 8-13.