面向机器人柑橘采摘的控制系统设计与试验

doi:10.13733/j.jcam.issn.2095-5553.2023.09.021

摘要/Abstract

摘要： 面向柑橘采摘，构建以上位机、RealSense Camera R200深度相机、VS-6556垂直多关节工业用机械臂、三指柔性手爪等组成的采摘机器人硬件平台。以Windows10为开发环境，采用librealsense相机软件开发工具包、OpenCV计算机视觉库、TensorFlowGPU和Keras深度学习框架、ORIN2机械臂控制软件开发工具包、Arduino IDE函数库以及SerialPort串口通信软件开发工具包等，研究基于深度相机、机械臂二次开发的采摘控制系统设计，包括视觉识别定位、手爪动作控制、机械臂运动控制以及采摘控制等模块的程序设计。采摘控制系统柑橘定位试验和柑橘采摘试验的测试结果显示，在实验室环境下面对随机布置的柑橘，视觉识别定位模块的平均定位精度误差为1.22cm，采摘过程中柑橘识别成功率达到100%，平均识别时间约为47ms，机器人柑橘采摘成功率达到80%，平均采摘时间约为15.2s，验证了采摘机器人平台控制系统程序的可行性，表明所开发的采摘控制系统能够正确、高效地完成整个柑橘采摘作业流程。

关键词: 采摘机器人, 柑橘采摘, 控制系统, 程序设计, 机器视觉

Abstract: For citrus picking, the hardware platform of picking robot composed of the upper computer, RealSense R200 depth Camera, VS-6556 vertical multijoint industrial manipulator and threefinger flexible paw was constructed. Windows10 as the development environment, librealsense camera software development kit, OpenCV computer vision library, TensorFlowGPU and Keras deep learning framework, ORIN2 manipulator control software development kit, Arduino IDE function library and SerialPort serial communication software development kit were adopted. The program design of picking control system based on depth camera and secondary development of mechanical arm included the module program design of visual identification and positioning, hand claw motion control, mechanical arm motion control and picking control. The pick control system citrus positioning and citrus picking experiment was conducted under the condition of random arrangement of citrus in the laboratory environment. The test result showed that the average positioning accuracy error of visual identification module was 1.22cm, in the process of picking oranges to identify the success rate of 100%, the average recognition time was about 47ms, citrus picking robot the success rate of 80%, the average picking time was about 15.2s. The test result verified the feasibility of the platform control system program of the picking robot, and showed that the control system could correctly and efficiently complete the whole citrus picking process.

Key words: picking robot, citrus picking, control system, program design, machine vision

中图分类号:

S666.2： TP242.6

王焱清, , 汤旸, 杨光友, . 面向机器人柑橘采摘的控制系统设计与试验[J]. 中国农机化学报, 2023, 44(9): 146-153.

Wang Yanqing, , Tang Yang, Yang Guangyou, . Design and experiment of control system for robot citrus picking[J]. Journal of Chinese Agricultural Mechanization, 2023, 44(9): 146-153.

参考文献

［1］　吕怡秋, 杨叶. 苹果采摘机械化发展研究［J］. 南方农机, 2020, 51(14): 7-8.
［2］　谭涛. 蔬果采摘机器人的研究进展与展望［J］. 现代农业研究, 2020, 26(5): 68-69.
Tan Tao. Research progress and prospect of vegetable and fruit harvesting robots ［J］. Modern Agriculture Research, 2020, 26(5): 68-69.
［3］　Roshanianfard A, Noguchi N. Characterization of pumpkin for a harvesting robot ［J］. IFAC-PapersOnLine, 2018, 51(17): 23-30.
［4］　Williams H A M, Jones M H, Nejati M, et al. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms ［J］. Biosystems Engineering, 2019, 181: 140-156.
［5］　Hayashi S, Shigematsu K, Yamamoto S, et al. Evaluation of a strawberryharvesting robot in a field test ［J］. Biosystems Engineering, 2010, 105(2): 160-171.
［6］　Zhao D, Lü J, Ji W, et al. Design and control of an apple harvesting robot ［J］. Biosystems Engineering, 2011, 110(2): 112-122.
［7］　占涛. 双臂协作农业采摘机器人若干关键技术研究［D］. 武汉: 湖北工业大学, 2017.
Zhan Tao. Research on several key technologies of agricultural picking robot with double manipulators cooperation ［D］. Wuhan: Hubei University of Technology, 2017.
［8］　苑进. 选择性收获机器人技术研究进展与分析［J］. 农业机械学报, 2020, 51(9): 1-17.
Yuan Jin. Research progress analysis of robotics selective harvesting technologies ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(9): 1-17.
［9］　顾宝兴, 姬长英, 王海青, 等. 智能移动水果采摘机器人设计与试验［J］. 农业机械学报, 2012, 43(6): 153-160.
Gu Baoxing, Ji Changying, Wang Haiqing, et al. Design and experiment of intelligent mobile fruit picking robot ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2012, 43(6): 153-160.
［10］　胡慧明. 基于双目视觉的棚室番茄采摘关键技术研究［D］. 武汉: 湖北工业大学, 2018.
Hu Huiming. Research on the key techniques of tomatoes harvesting from greenhouse based on binocular stereo vision ［D］. Wuhan: Hubei University of Technology, 2018.
［11］　吕石磊, 卢思华, 李震, 等. 基于改进YOLOv3-LITE轻量级神经网络的柑橘识别方法［J］. 农业工程学报, 2019, 35(17): 205-214.
Lü Shilei, Lu Sihua, Li Zhen, et al. Orange recognition method using improved YOLOv3-LITE lightweight neural network ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(17): 205-214.
［12］　Onishi Y, Yoshida T, Kurita H, et al. An automated fruit harvesting robot by using deep learning ［J］. Robomech Journal, 2019, 6: 13-20.
［13］　杨长辉, 刘艳平, 王毅, 等. 自然环境下柑橘采摘机器人识别定位系统研究［J］. 农业机械学报, 2019, 50(12): 14-22, 72.
Yang Changhui, Liu Yanping, Wang Yi, et al. Research and experiment on recognition and location system for citrus picking robot in natural environment ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(12): 14-22, 72.
［14］　邓小雷, 罗忠祎, 庞景权, 等. 仿生无损吸取式苹果采摘装置的设计与试验［J］. 中国农业大学学报, 2019, 24(10): 100-108.
Deng Xiaolei, Luo Zhongyi, Pang Jingquan, et al. Design and experiment of bionic nondestructive handheld suction apple picker ［J］. Journal of China Agricultural University, 2019, 24(10): 100-108.
［15］　刘凡, 杨光友, 杨康. 农业采摘机器人柔性机械手研究［J］. 中国农机化学报, 2019, 40(3): 173-178.
Liu Fan, Yang Guangyou, Yang Kang. Research on flexible manipulator for agricultural picking robot ［J］. Journal of Chinese Agricultural Mechanization, 2019, 40(3): 173-178.
［16］　吴群彪, 许侃雯, 张洪源, 等. 草莓采摘机器人控制系统的设计［J］. 包装与食品机械, 2021, 39(2): 58-62.
Wu Qunbiao, Xu Kanwen, Zhang Hongyuan, et al. Design of strawberry picking robot control system ［J］. Packaging and Food Machinery, 2021, 39(2): 58-62.
［17］　熊俊涛, 叶敏, 邹湘军, 等. 多类型水果采摘机器人系统设计与性能分析［J］. 农业机械学报, 2013, 44(S1): 230-235.
Xiong Juntao, Ye Min, Zou Xiangjun, et al. System design and performance analysis on multitype fruit harvesting robot ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2013, 44(S1): 230-235.
［18］　刘静, 林冲, 郭世财, 等. 柑橘类水果采摘机器的设计与研究［J］. 包装工程, 2019, 40(17): 56-62.
Liu Jing, Lin Chong,Guo Shicai, et al. Design and research of citrus fruit picking machine ［J］. Packaging Engineering, 2019, 40(17): 56-62.
［19］　Redmon J, Farhadi A. YOLOv3: An incremental improvement ［J］. arXiv eprints, 2018.
［20］　Kaiming H, Xiangyu Z, Shaoqing R, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition ［J/OL］. https: ∥arxiv.org/abs/1406.4729v4, 2015-04-23.
［21］　Howard A, Sandler M, Chu G, et al. Searching for MobileNetV3［J/OL］. https: ∥arxiv.org/abs/1905.02244, 2019-11-20.
［22］　Zheng Z, Wang P, Liu W, et al. DistanceIoU loss: Faster and better learning for bounding box regression ［J］. arXiv, 2019.

[1]	侯义锋, , 钱俊, , 王梁, 贺杰, 闭耀锋. 砂糖橘采摘机器人分拣机构设计[J]. 中国农机化学报, 2023, 44(9): 183-189.
[2]	王大明, 何逸霏, 李华英, 苟于江, 何辉波. 柑橘采摘机器人图像识别算法研究[J]. 中国农机化学报, 2023, 44(9): 222-226.
[3]	姜来, 王文娣, 霍晓静, 王辉, 唐娟, 李丽华, (. 死鸡识别机器人系统设计与试验[J]. 中国农机化学报, 2023, 44(8): 81-87.
[4]	刘浩, 孟令峰, 王松峰, 刘自畅, 杜海娜, 孙福山. 基于机器视觉的烤烟鲜烟成熟度判别模型优选[J]. 中国农机化学报, 2023, 44(8): 118-124.
[5]	舒卓, 陈立平, 陈梅香, 张瑞瑞, 郭新宇, 温维亮. 单目多视角图像的美国白蛾三维模型重建系统与试验[J]. 中国农机化学报, 2023, 44(8): 168-173.
[6]	刘雯雯, 孙裕晶, 姜树辉, 姜鹏. 遮挡情况下的行人检测方法研究[J]. 中国农机化学报, 2023, 44(7): 179-186.
[7]	周红标, 朱亚文, 刘晓洋, 张国良, 李边豪. 基于机器视觉的水稻印刷播种机成种率检测[J]. 中国农机化学报, 2023, 44(7): 194-199.
[8]	王亚林, 印祥, 杨爽, 姚海斌, 马文鹏, 金诚谦, . 基于PWM的电控精量喷雾控制系统设计与试验[J]. 中国农机化学报, 2023, 44(5): 81-88.
[9]	陈宇, 杨创, 唐维东, 张九通, 蒋猛. 小蚕共育自动饲育机控制系统设计与试验[J]. 中国农机化学报, 2023, 44(3): 55-63.
[10]	熊钦, 肖丽萍, 蔡金平, 董伟, 张鸿, 陶中岩. 基于物联网的果园药水肥一体化控制系统设计与实现[J]. 中国农机化学报, 2023, 44(3): 73-81.
[11]	罗山, 侯俊涛, 郑彬. 基于机器视觉的石榴品质自动分级方法[J]. 中国农机化学报, 2023, 44(3): 117-122.
[12]	吴俊鹏, 黄光文, 李君, . 水果病害视觉与光谱检测技术研究现状及展望[J]. 中国农机化学报, 2023, 44(2): 112-118.
[13]	王宜雷, 宋智, 刘永华. 基于水动力学模型的渠道流量控制系统研究[J]. 中国农机化学报, 2023, 44(2): 195-199.
[14]	李志臣, 凌秀军, 李鸿秋. 基于机器视觉的槟榔分级方法研究[J]. 中国农机化学报, 2023, 44(1): 137-141.
[15]	姬丽雯, , 刘永华, , 高菊玲, , 吴丹, . 温室草莓采摘机器人设计与试验[J]. 中国农机化学报, 2023, 44(1): 192-198.