水果采摘机器人目标识别与采摘点定位研究进展

doi:10.13733/j.jcam.issn.2095-5553.2025.05.016

摘要/Abstract

摘要： 水果采摘机器人对实现水果自动化采摘具有重要意义，视觉系统是采摘机器人研究的关键。为此，对近几年国内外采摘机器人视觉系统的关键技术研究工作进行总结。按照采摘视觉系统的技术路线，论述采摘视觉系统图像采集技术；总结水果目标识别领域常用的目标识别算法：单阶段算法、二阶段算法、基于深度学习的语义分割和实例分割算法及其改进算法，总结在复杂环境下识别技术应用的关键；并以采摘点的获取方法为轴心，选择水果目标定位的硬件系统和软件算法。最后探讨采摘视觉系统目前面临算法、硬件、适用环境等方面的局限性，提出未来应该关注多信息融合、深度学习技术、复杂环境下视觉系统等方向的研究，为水果采摘机器人的研究提供借鉴和指导价值。

关键词: 水果, 采摘机器人, 机器视觉, 采摘点定位, 目标识别

Abstract: Fruit picking robot is of great significance to realize automatic fruit picking, and vision system is the key to the research of fruit picking robot. Therefore, the research work on the key technologies of vision system of picking robot at home and abroad in recent years is summarized. According to the technical route of the picking vision system, the image acquisition technology of the picking vision system is discussed. This paper summarizes the common target recognition algorithms in the field of fruit target recognition, including singlestage algorithm, twostage algorithm, semantic segmentation and instance segmentation algorithms based on deep learning and their improvements, and summarizes the application of recognition technology in complex environment. The hardware system and software algorithm of fruit target location are summarized based on the method of picking point acquisition. Finally, the paper discusses the limitations of the picking vision system in algorithm, hardware and applicable environment, and proposes that the future research should focus on multiinformation fusion, deep learning technology and vision system in complex environment, so as to provide reference and guidance value for the research of fruit picking robots.

Key words: fruit, harvesting robots, machine vision, picking point localization, target recognition

中图分类号:

石国照, 张富贵, , 苟园旻, 郑乐, 蔡景勇, 冯池. 水果采摘机器人目标识别与采摘点定位研究进展[J]. 中国农机化学报, 2025, 46(5): 115-124.

Shi Guozhao, Zhang Fugui, , Gou Yuanmin, Zheng Le, Cai Jingyong, Feng Chi. Research progress on target recognition and picking point localization of fruit picking robots [J]. Journal of Chinese Agricultural Mechanization, 2025, 46(5): 115-124.

参考文献

［1］　Zhao J, Tsuchikawa S, Ma T, et al. Modal analysis and experiment of a lycium barbarum L. Shrub for efficient vibration harvesting of fruit ［J］. Agriculture, 2021, 11（6）: 519.
［2］　Lehnert C, English A, McCool C, et al. Autonomous sweet pepper harvesting for protected cropping systems ［J］. IEEE Robotics and Automation Letters, 2017, 2（2）: 872-879.
［3］　Lehnert C, McCool C, Sa I, et al. Performance improvements of a sweet pepper harvesting robot in protected cropping environments ［J］. Journal of Field Robotics, 2020, 37（7）: 1197-1223.
［4］　Williams H A M, Jones M H, Nejati M, et al. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms ［J］. Biosystems Engineering, 2019, 181: 140-156.
［5］　Williams H, Ting C, Nejati M, et al. Improvements to and largescale evaluation of a robotic kiwifruit harvester ［J］. Journal of Field Robotics, 2020, 37（2）: 187-201.
［6］　Feng Q, Zou W, Fan P, et al. Design and test of robotic harvesting system for cherry tomato ［J］. International Journal of Agricultural and Biological Engineering, 2018, 11（1）: 96-100.
［7］　Yin H, Sun Q, Ren X, et al. Development, integration, and field evaluation of an autonomous citrusharvesting robot ［J］. Journal of Field Robotics, 2023， 40（6): 1363-1387.
［8］　Ling X, Zhao Y, Gong L, et al. Dualarm cooperation and implementing for robotic harvesting tomato using binocular vision ［J］. Robotics and Autonomous Systems, 2019, 114: 134-143.
［9］　Silwal A, Davidson J R, Karkee M, et al. Design, integration, and field evaluation of a robotic apple harvester ［J］. Journal of Field Robotics, 2017, 34（6）: 1140-1159.
［10］　Kang H, Zhou H, Chen C. Visual perception and modeling for autonomous apple harvesting ［J］. IEEE Access, 2020, 8: 62151-62163.
［11］　Tafuro A, Adewumi A, Parsa S, et al. Strawberry picking point localization ripeness and weight estimation ［C］. International Conference on Robotics and Automation. IEEE, 2022: 2295-2302.
［12］　Zhong Z, Xiong J, Zheng Z, et al. A method for litchi picking points calculation in natural environment based on main fruit bearing branch detection ［J］. Computers and Electronics in Agriculture, 2021, 189: 106398.
［13］　朱智惟, 单建华, 余贤海，等. 基于YOLOv5s的番茄采摘机器人目标检测技术［J］.传感器与微系统, 2023, 42（6）: 129-132.Zhu Zhiwei, Shan Jianhua, Yu Xianhai, et al.Target detection technology of tomato picking robot base on YOLOv5s ［J］.Transducer and Microsystem Technologys,2023,42（6）:129-132.
［14］　Wachs J P, Stern H I, Burks T, et al. Low and highlevel visual featurebased apple detection from multimodal images ［J］. Precision Agriculture, 2010, 11: 717-735.
［15］　熊俊涛, 邹湘军, 王红军，等. 基于Retinex图像增强的不同光照条件下的成熟荔枝识别［J］. 农业工程学报, 2013, 29（12）: 170-178.
Xiong Juntao, Zou Xiangjun, Wang Hongjun, et al. Recognition of ripe litchi in different illumination conditions based on Retinex image enhancement ［J］. Tvansactions of the Chinese Society of Agricultural Engineering,2013, 29（12）: 170－178.
［16］　Lü J, Wang Y, Xu L, et al. A method to obtain the nearlarge fruit from apple image in orchard for singlearm apple harvesting robot ［J］. Scientia Horticulturae, 2019, 257: 108758.
［17］　Li M. Research on color correction method of greenhouse tomato plant image based on high dynamic range imaging ［J］. INMATEHAgricultural Engineering, 2021, 64（2）.
［18］　Xu Z F, Jia R S, Liu Y B, et al. Fast method of detecting tomatoes in a complex scene for picking robots ［J］. IEEE Access, 2020, 8: 55289-55299.
［19］　Luo Z, Yu H, Zhang Y. Pine cone detection using boundary equilibrium generative adversarial networks and improved YOLOv3model ［J］. Sensors, 2020, 20（16）: 4430.
［20］　Lu J, Sang N. Detecting citrus fruits and occlusion recovery under natural illumination conditions ［J］. Computers and Electronics in Agriculture, 2015, 110: 121-130.
［21］　司永胜, 乔军, 刘刚，等.苹果采摘机器人果实识别与定位方法［J］. 农业机械学报, 2010,41（9）: 148-153.
Si Yongsheng, Qiao Jun, Liu Gang, et al.Recognition and location of fruits for apple harvesting robot ［J］.Transactions of the Chinese Society for Agricultural Machinery,2010, 41（9）: 148-153.
［22］　齐锐丽, 陈曼龙, 杨宗浩，等. 基于HSV模型与改进的OTSU算法花椒图像分割［J］.中国农机化学报, 2019, 40（11）: 155-160.
Qi Ruili, Chen Manlong, Yang Zonghao, et al. Image segmentation of Sichuan pepper based on HSV model and improved OTSU algorithm ［J］. Journal of Chinese Agricultural Mechanization, 2019, 40（11）: 155-160.
［23］　熊俊涛, 邹湘军, 陈丽娟，等. 基于机器视觉的自然环境中成熟荔枝识别［J］. 农业机械学报, 2011, 42（9）: 162-166.
Xiong Juntao,Zou Xiangjun,Chen Lijuan, et al.Recognition of mature litchi in natural environment based on machine vision ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2011, 42（9）: 162-166.
［24］　Lin G, Zou X. Citrus segmentation for automatic harvester combined with adaboost classifier and LeungMalik filter bank ［J］. IFACPapersOnLine, 2018, 51（17）: 379-383.
［25］　Wu J, Zhang B, Zhou J, et al. Automatic recognition of ripening tomatoes by combining multifeature fusion with a bilayer classification strategy for harvesting robots ［J］. Sensors, 2019, 19（3）: 612.
［26］　Zou Z, Chen K, Shi Z, et al. Object detection in 20 years: A survey ［J］. Proceedings of the IEEE, 2023，111（3）：257-276.
［27］　Gao F, Fu L, Zhang X, et al. Multiclass fruitonplant detection for apple in SNAP system using Faster R—CNN ［J］. Computers and Electronics in Agriculture, 2020, 176: 105634.
［28］　Sa I, Ge Z, Dayoub F, et al. Deepfruits: A fruit detection system using deep neural networks ［J］. Sensors, 2016, 16（8）: 1222.
［29］　闫建伟, 赵源, 张乐伟, 等. 改进Faster—RCNN自然环境下识别刺梨果实［J］. 农业工程学报, 2019, 35（18）: 143-150.
Yan Jianwei, Zhao Yuan, Zhang Lewei, et al. Recognition of Rosa roxbunghii in natural environment based on improved Faster—RCNN ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35（18）: 143-150.
［30］ Tu S, Pang J, Liu H, et al. Passion fruit detection and counting based on multiple scale faster R—CNN using RGB—D images ［J］. Precision Agriculture, 2020, 21: 1072-1091.
［31］　Tian Y, Yang G, Wang Z, et al. Apple detection during different growth stages in orchards using the improved YOLO—V3 model ［J］. Computers and Electronics in Agriculture, 2019, 157: 417-426.
［32］　Yu Y, Zhang K, Liu H, et al. Realtime visual localization of the picking points for a ridgeplanting strawberry harvesting robot ［J］. IEEE Access, 2020, 8: 116556-116568.
［33］　Fu L, Wu F, Zou X, et al. Fast detection of banana bunches and stalks in the natural environment based on deep learning ［J］. Computers and Electronics in Agriculture, 2022, 194: 106800.
［34］　刘洁, 李燕, 肖黎明, 等. 基于改进YOLOv4模型的橙果识别与定位方法［J］. 农业工程学报, 2022, 38（12）: 173-182.
Liu Jie, Li Yan, Xiao Liming, et al. Recognition and location method of orange based on improved YOLOv4 model ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38（12）: 173-182.
［35］　Cao Z, Mei F, Zhang D, et al. Recognition and detection of persimmon in a natural environment based on an improved YOLOv5 model ［J］. Electronics, 2023, 12（4）: 785.
［36］　彭红星, 黄博, 邵园园, 等. 自然环境下多类水果采摘目标识别的通用改进SSD模型［J］.农业工程学报, 2018, 34（16）: 155-162.
Peng Hongxing, Huang Bo, Shao Yuanyuan, et al. General improved SSD model for picking object recognition of multiple fruits in natural environment ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34（16）: 155-162.
［37］　Liu G, Nouaze J C, Touko Mbouembe P L, et al. YOLO-tomato: A robust algorithm for tomato detection based on YOLOv3 ［J］. Sensors, 2020, 20（7）: 2145.
［38］　Lin G, Tang Y, Zou X, et al. Guava detection and pose estimation using a lowcost RGB—D sensor in the field ［J］. Sensors, 2019, 19（2）: 428.
［39］　李艳文, 左朝阳, 王登奎, 等. 基于改进型SegNet的苹果采摘点分割算法研究［J］.燕山大学学报,2022, 46（5）: 455-460.
Li Yanwen, Zuo Chaoyang, Wang Dengkui, et al. Research on apple picking point segmentation algorithm based on improved SegNet ［J］.Journal of Yanshan University, 2022, 46（5）: 455-460.
［40］　Li J, Tang Y, Zou X, et al. Detection of fruitbearing branches and localization of litchi clusters for visionbased harvesting robots ［J］. IEEE Access, 2020, 8: 117746-117758.
［41］　LópezBarrios J D, Escobedo Cabello J A, GómezEspinosa A, et al. Green sweet pepper fruit and peduncle detection using Mask R—CNN in greenhouses ［J］. Applied Sciences, 2023, 13（10）: 6296.
［42］　Zhang T, Huang Z, You W, et al. An autonomous fruit and vegetable harvester with a lowcost gripper using a 3D sensor ［J］. Sensors, 2019, 20（1）: 93.
［43］　Li H, Lee W S, Wang K. Immature green citrus fruit detection and counting based on fast normalized cross correlation （FNCC） using natural outdoor colour images ［J］. Precision Agriculture, 2016, 17: 678-697.
［44］　Bac C W, Hemming J, Van Henten E J. Robust pixelbased classification of obstacles for robotic harvesting of sweetpepper ［J］. Computers and Electronics in Agriculture, 2013, 96: 148-162.
［45］　田有文,赖兴涛,张芳,等.基于高光谱成像的苹果果梗完整性识别方法研究［J］.沈阳农业大学学报, 2018, 49（2）: 234-241.
Tian Youwen, Lai Xingtao, Zhang Fang, et al. Research on identification method of apple stem integrity based on hyperspectral imaging ［J］. Journal of Shenyang Agricultural University, 2018, 49（2）: 234-241.
［46］　Liu X, Zhao D, Jia W, et al. A method of segmenting apples at night based on color and position information ［J］. Computers and Electronics in Agriculture, 2016, 122: 118-123.
［47］　Xiong J, Lin R, Liu Z, et al. The recognition of litchi clusters and the calculation of picking point in a nocturnal natural environment ［J］. Biosystems Engineering, 2018, 166: 44-57.
［48］　Xiang R. Image segmentation for whole tomato plant recognition at night ［J］. Computers and Electronics in Agriculture, 2018, 154: 434-442.
［49］　Liang C, Xiong J, Zheng Z, et al. A visual detection method for nighttime litchi fruits and fruiting stems ［J］. Computers and Electronics in Agriculture, 2020, 169: 105192.
［50］　吕继东, 赵德安, 姬伟,等. 采摘机器人振荡果实动态识别［J］.农业机械学报, 2012, 43（5）:173-178, 196.
Lü Jidong, Zhao Dean, Ji Wei, et al. Dynamic recognition of oscillating fruit for harvesting robot ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2012, 43（5）: 173-178, 196.
［51］　吕继东, 赵德安, 姬伟,等. 苹果采摘机器人对振荡果实的快速定位采摘方法［J］.农业工程学报, 2012, 28（13）: 48-53.
Lü Jidong, Zhao Dean, Ji Wei, et al. Fast positioning method of apple harvesting robot for oscillating fruit ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28（13）: 48－53.
［52］　赵德安, 沈甜, 陈玉, 等. 苹果采摘机器人快速跟踪识别重叠果实［J］. 农业工程学报, 2015, 31（2）: 22-28.
Zhao Dean,Shen Tian,Chen Yun,et al. Fast tracking and identification method of target fruit for apple picking robots ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31（2）: 22-28.
［53］　苟园旻, 闫建伟, 张富贵, 等. 水果采摘机器人视觉系统与机械手研究进展［J］.计算机工程与应用,2023, 59（9）: 13-26.
Gou yuanmin, Yan Jianwei, Zhang Fugui, et al. Research progress on vision system and manipulator of fruit picking robot ［J］. Computer Engineering and Applications, 2023, 59（9）: 13-26.
［54］　Sun L, Cai J R, Zhao J W. A vision system based on TOF 3D imaging technology applied to robotic citrus harvesting ［J］. Intelligent Automation & Soft Computing, 2015, 21（3）: 345-354.
［55］　Benavides M, CantónGarbín M, SánchezMolina J A, et al. Automatic tomato and peduncle location system based on computer vision for use in robotized harvesting ［J］. Applied Sciences, 2020, 10（17）: 5887.
［56］　Yaguchi H, Nagahama K, Hasegawa T, et al. Development of an autonomous tomato harvesting robot with rotational plucking gripper ［C］. 2016 IEEE/RSJ international conference on intelligent robots and systems （IROS）. IEEE, 2016: 652-657.
［57］　Jun J, Kim J, Seol J, et al. Towards an efficient tomato harvesting robot: 3D perception, manipulation, and endeffector ［J］. IEEE Access, 2021, 9: 17631-17640.
［58］　Lehnert C, Sa I, McCool C, et al. Sweet pepper pose detection and grasping for automated crop harvesting ［C］. International Conference on Robotics and Automation, 2016: 2428-2434.
［59］　Eizentals P, Oka K. 3D pose estimation of green pepper fruit for automated harvesting ［J］. Computers and Electronics in Agriculture, 2016, 128: 127-140.
［60］　Ge Y, Xiong Y, From P J. Symmetrybased 3D shape completion for fruit localisation for harvesting robots ［J］. Biosystems Engineering, 2020, 197: 188-202.
［61］　Yu Y, Zhang K, Yang L, et al. Fruit detection for strawberry harvesting robot in nonstructural environment based on Mask R—CNN ［J］. Computers and Electronics in Agriculture, 2019, 163: 104846.
［62］　朱立学, 赖颖杰, 张世昂, 等. 基于改进U—Net的火龙果采摘图像分割和姿态估计方法［J］. 农业机械学报, 2023, 54(11): 180-188.
Zhu Lixue, Lai Yingjie, Zhang Shiang, et al. Image segmentation and pose estimation method for pitaya picking robot based on enhanced U—Net ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(11): 180-188.

[1]	张俊宁, 闫英, 胡欢, 毕泽洋, 邢宇. 基于Sim—YOLO的复杂背景下樱桃番茄多目标检测[J]. 中国农机化学报, 2025, 46(3): 238-245.
[2]	邱兆鑫, , 郭辉, 高国民, 武天伦, 陈海洋. 红花采摘机器人末端采摘装置设计与试验[J]. 中国农机化学报, 2025, 46(3): 271-276.
[3]	李涛, 买买提明⋅艾尼, 古丽巴哈尔⋅托乎提, 杨佳雨. 基于改进YOLOv8的小棚架下无核白葡萄果梗识别[J]. 中国农机化学报, 2025, 46(2): 259-263.
[4]	钟沅, 陈泽鸿, 郑君彬, 宋淑然, 孙道宗, 刘洪山. 基于机器视觉的在线果树靶标识别装置[J]. 中国农机化学报, 2025, 46(1): 164-170.
[5]	李春雨, 郭肖琴, 杨晶晶, 李忠华. 基于深度学习的水果图像识别[J]. 中国农机化学报, 2025, 46(1): 198-203.
[6]	熊明明 , 李晓娟 , . 基于图像与深度信息融合的西梅果实识别定位方法[J]. 中国农机化学报, 2024, 45(9): 172-177.
[7]	桑一男 , 徐增莱 , 汪琼 , 葛海涛 , 王殿广 . 一种轮式采花机器人设计——以黄蜀葵为例[J]. 中国农机化学报, 2024, 45(9): 202-208.
[8]	付晓鸽, 李涵, 左治江, 杜铮 . 基于 Mask R-CNN的复杂环境下辣椒识别方法研究[J]. 中国农机化学报, 2024, 45(9): 215-219.
[9]	王明龙 , 徐垚 , 张智浩 , 朱立学 , 林桂潮 , . 茶叶智能机械化采摘技术与装备研究进展[J]. 中国农机化学报, 2024, 45(9): 305-310.
[10]	陈蒙, 陈丰, 张华, 丁文芹, 占才学. 椭球型果蔬柔性采摘灵巧手设计[J]. 中国农机化学报, 2024, 45(8): 32-37.
[11]	何梁, 薛龙, 曾瑜琦, 刘木华, 杨君, 黎静. 莲蓬采摘末端执行器设计[J]. 中国农机化学报, 2024, 45(8): 38-42.
[12]	柳军, 孔杰, 皮杰, 周成钢. 果蔬分拣设备研究现状及发展趋势[J]. 中国农机化学报, 2024, 45(8): 120-125.
[13]	苑迎春, 张傲, 何振学, 张若晨, 雷浩. 基于改进YOLOv4-tiny的果园复杂环境下桃果实实时识别[J]. 中国农机化学报, 2024, 45(8): 254-261.
[14]	王雷, 钟康生, 郭小宝, 盛斌, 肖波, 荆磊. 基于机器视觉的陈皮热泵干燥含水率预测模型研究[J]. 中国农机化学报, 2024, 45(7): 97-103.
[15]	张延军, 赵建鑫, . 基于多路神经网络的黄花菜关节点检测算法研究[J]. 中国农机化学报, 2024, 45(7): 228-234.