基于GoogLeNet的玉米籽粒破损及霉变在线辨识方法

doi:10.13733/j.jcam.issn.2095-5553.2023.10.013

摘要/Abstract

摘要： 针对采集的玉米籽粒图像存在冗余信息多、目标区域占比小、目标位置具有随机性等问题，提出一种基于颜色空间(HSV)阈值分割的玉米籽粒图像切片处理方法，提高玉米籽粒破损及霉变的辨识准确率。首先，确定分割阈值，提取出玉米籽粒轮廓，根据轮廓的最小外接矩形框顶点坐标，确定裁剪坐标并裁剪图像；其次，获得未经图像切片处理与切片处理的GoogLeNet模型权重，在GoogLeNet第一、二、三层卷积及inception5b模块后使用GradCAM可视化方法，对不同卷积层所提取的特征进行可视化解释；最后，结合验证集准确率评价两组模型对完好、破损以及霉变玉米籽粒的特征提取能力。结果表明，本文提出的图像切片处理方法可将验证集准确率提高到93.74%，相较于未经图像切片处理的数据集，在GoogLeNet上验证集准确率提升7.99%；借助GradCAM可视化方法显示模型关注区域，可视化解释网络提取的特征，验证该方法的有效性，为玉米籽粒图像预处理及辨识分类提供新思路。

关键词: 玉米籽粒, 阈值分割, 破损, 霉变, 卷积

Abstract: Aiming at the problems such as excessive redundant information, small proportion of target area and randomness of target location in the collected corn grain images, a new corn grain image slicing method based on color space (HSV) threshold segmentation was proposed in this paper, which improved the recognition accuracy of damaged and mildewed corn grain. Firstly, the segmentation threshold was determined, the corn grain contour was extracted, and the cropping coordinates were determined and the image was clipped according to the vertex coordinates of the minimum external rectangle frame of the contour. Secondly, the weight of GoogLeNet model without image slicing processing and with image slicing processing were obtained. After the first, second and third convolution layer and inception5b module of GoogLeNet, the Grad-CAM visualization method was used to visualize the features extracted from different convolutional layers. Finally, based on the accuracy of verification set, the ability of the two models to extract intact, damaged and mildewed corn grain features was evaluated. The results showed that the method proposed in this paper could improve the verification set accuracy to 93.74%, which was 7.99% higher than that of the data set without image slicing processing on GoogLeNet. The models concerned areas were displayed by the Grad-CAM visualization method, and the features extracted from the network were interpreted visually, which verified the effectiveness of this method and provided a new idea for the pretreatment and identification of corn kernel image.

Key words: corn kernel, threshold segmentation, damaged, mildewed, convolution

中图分类号:

S238： TP391.4

林杰, 王发赢, 姚艳春, 崔春晓, 盛振哲, 曲殿伟. 基于GoogLeNet的玉米籽粒破损及霉变在线辨识方法[J]. 中国农机化学报, 2023, 44(10): 87-92.

Lin Jie, , Wang Faying, , Yao Yanchun, , , Cui Chunxiao, , Sheng Zhenzhe, , Qu Dianwei. Online identification method of corn kernel damage and mildew based on GoogLeNet[J]. Journal of Chinese Agricultural Mechanization, 2023, 44(10): 87-92.

参考文献

［1］　蔡桂全，陶建平. 基于细菌觅食优化多核支持向量机的作物生长环境控制［J］. 济南大学学报(自然科学版), 2023, 37(3): 303-308.
Cai Guiquan, Tao Jianping.Crop growth environment control based on bacterial foraging optimization multikernel support vector machine ［J］. Journal of University of Jinan (Science and Technology), 2023, 37(3): 303-308.
［2］　王春雷, 卢彩云, 李洪文, 等. 基于支持向量机的玉米根茬行图像分割［J］. 农业工程学报, 2021, 37(16): 117-126.
Wang Chunlei, Lu Caiyun, Li Hongwen, et al. Image segmentation of maize stubble row based on SVM ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(16): 117-126.
［3］　孙玉婷, 杨红云, 王映龙, 等. 基于支持向量机的水稻叶面积测定［J］. 江苏农业学报, 2018, 34(5): 1027-1035.Sun Yuting, Yang Hongyun, Wang Yinglong, et al. Determination of rice leaf area based on support vector machine ［J］. Jiangsu Journal of Agricultural Sciences, 2018, 34(5): 1027-1035.
［4］　汪斌斌, 杨贵军, 杨浩, 等. 基于YOLO_X和迁移学习的无人机影像玉米雄穗检测［J］. 农业工程学报, 2022, 38(15): 53-62.
Wang Binbin, Yang Guijun, Yang Hao, et al. UAV images for detecting maize tassel based on YOLO_X and transfer learning ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(15): 53-62.
［5］　顾金梅, 吴雪梅, 龙曾宇, 等. 基于BP神经网络的烟叶颜色自动分级研究［J］. 中国农机化学报, 2016, 37(4)： 110-114.
Gu Jinmei, Wu Xuemei, Long Cengyu, et al. Automatic classification research on tobacco leaf color based on BP neural network［J］. Journal of Chinese Agricultural Mechanization, 2016, 37(4): 110-114.
［6］　任浩, 李丽, 卢世博, 等. 基于深度学习的复杂自然环境下桑树枝干识别方法［J］. 中国农机化学报, 2023, 44(2)： 182-188.
Ren Hao, Li Li, Lu Shibo, et al. Identification method of mulberry tree branches in complex natural environments based on deep learning［J］. Journal of Chinese Agricultural Mechanization, 2023, 44(2): 182-188.
［7］　Alex K, Ilya S, Hinton G. Imagenet classification with deep convolutional neural networks［J］. Communications of the Association for Computing Machinery, 2017, 60(6): 84-90.
［8］　Zeiler M D, Fergus R. Visualizing and understanding convolutional networks［C］. European Conference on Computer Vision, 2014.
［9］　王立扬, 张瑜, 沈群, 等. 基于改进型LeNet-5的苹果自动分级方法［J］. 中国农机化学报, 2020, 41(7)： 105-110.
Wang Liyang, Zhang Yu, Shen Qun, et al. Automatic detecting and grading method of apples based on improved LeNet-5［J］. Journal of Chinese Agricultural Mechanization, 2020, 41(7): 105-110.
［10］　张瑞青, 李张威, 郝建军, 等. 基于迁移学习的卷积神经网络花生荚果等级图像识别［J］. 农业工程学报, 2020, 36(23)： 171-180.
Zhang Ruiqing, Li Zhangwei, Hao Jianjun, et al. Image recognition of peanut pod grades based on transfer learning with convolutional neural network ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(23): 171-180.
［11］　张建华, 孔繁涛, 吴建寨, 等. 基于改进VGG卷积神经网络的棉花病害识别模型［J］. 中国农业大学学报, 2018, 23(11)： 161-171.
Zhang Jianhua, Kong Fantao, Wu Jianzhai, et al. Cotton disease identification model based on improved VGG convolution neural network ［J］. Journal of China Agricultural University, 2018, 23(11): 161-171.
［12］　王林柏, 张博, 姚竟发, 等. 基于卷积神经网络马铃薯叶片病害识别和病斑检测［J］. 中国农机化学报, 2021, 42(11)： 122-129.
Wang Linbai, Zhang Bo, Yao Jingfa, et al. Potato leaf disease recognition and potato leaf disease spot detection based on convolutional neural network［J］. Journal of Chinese Agricultural Mechanization, 2021, 42(11): 122-129.
［13］　陈文博, 刘昌华, 刘春苔, 等. 基于GoogLeNet的稻米品种识别与碎米检测［J］. 中国粮油学报, 2023, 38(2)： 146-152.
Chen Wenbo, Liu Changhua, Liu Chuntai, et al. Identification rice varieties and broken rice based on GoogLeNet ［J］. Journal of the Chinese Cereals and Oils Association, 2023, 38(2): 146-152.
［14］　Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition［J］. Computer Science, 2014, 34(2): 1409-1422.
［15］　Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions ［C］. 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015.
［16］　侯俊铭, 姚恩超, 朱红杰. 基于卷积神经网络的蓖麻种子损伤分类研究［J］. 农业机械学报, 2020, 51(S1)： 440-449.
Hou Junming, Yao Enchao, Zhu Hongjie. Classification of castor seed damage based on convolutional neural network ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(S1): 440-449.
［17］　苏宝峰, 沈磊, 陈山, 等. 基于注意力机制的葡萄品种多特征分类方法［J］. 农业机械学报, 2021, 52(11)： 226-233, 252.
Su Baofeng, Shen Lei, Chen Shan, et al. Multifeatures identification of grape cultivars based on attention mechanism ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 52(11): 226-233, 252.
［18］　徐岩, 刘林, 李中远, 等. 基于卷积神经网络的玉米品种识别［J］. 江苏农业学报, 2020, 36(1)： 18-23.
Xu Yan, Liu Lin, Li Zhongyuan, et al. Recognition of maize varieties based on convolutional neural network ［J］. Jiangsu Journal of Agricultural Sciences, 2020, 36(1): 18-23.
［19］　权龙哲, 王建宇, 王旗, 等. 基于电磁振动与卷积神经网络的玉米品质精选装置［J］. 江苏大学学报(自然科学版), 2020, 41(3)： 288-293, 313.
Quan Longzhe, Wang Jianyu, Wang Qi, et al. Classification method of corn quality selection based on electromagnetic vibration and convolutional neural network ［J］. Journal of Jiangsu University (Natural Science Edition), 2020, 41(3): 288-293, 313.
［20］　高震宇, 王安, 刘勇, 等. 基于卷积神经网络的鲜茶叶智能分选系统研究［J］. 农业机械学报, 2017, 48(7)： 53-58.
Gao Zhenyu, Wang An, Liu Yong, et al. Intelligent freshtealeaves sorting system research based on convolution neural network［J］. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48(7): 53-58.
［21］　曹旨昊. 基于深度学习的图像推荐算法研究与实现［D］. 泰安：山东农业大学, 2020.Cao Zhihao. Research and implementation of image recommendation algorithms based on deep learning ［D］. Taian: Shandong Agricultural University, 2020.
［22］　张旭. 基于深度学习的苹果叶部病害识别及应用［D］. 沈阳：沈阳农业大学, 2023.Zhang Xu. Recognition and application of apple leaf diseases based on deep learning ［D］. Shenyang: Shenyang Agricultural University, 2023.

[1]	梁万杰, 曹静, 孙传亮, 曹宏鑫, 张文宇, . 基于深度学习的农作物病害识别系统研发[J]. 中国农机化学报, 2023, 44(9): 169-175.
[2]	杨波, 何金平, 张立娜. 基于改进SPP-x的YOLOv5神经网络水稻叶片病害识别检测[J]. 中国农机化学报, 2023, 44(9): 190-197.
[3]	汪健, 梁兴建, 雷刚. 基于深度残差网络与迁移学习的水稻病虫害图像识别[J]. 中国农机化学报, 2023, 44(9): 198-204.
[4]	余胜, 谢莉. 基于迁移学习和卷积视觉转换器的农作物病害识别研究[J]. 中国农机化学报, 2023, 44(8): 191-197.
[5]	马丽, 周巧黎, 赵丽亚, 胡远辉. 基于深度学习的番茄叶片病害分类识别研究[J]. 中国农机化学报, 2023, 44(7): 187-193.
[6]	李宗南, 蒋怡, 王思, 李源洪, 黄平, 魏鹏. 基于YOLOv5模型的飞蓬属入侵植物目标检测[J]. 中国农机化学报, 2023, 44(7): 200-206.
[7]	李平, 马玉琨, 李艳翠, 冯继克, 赵明富. 基于迁移学习的小麦籽粒品种识别研究[J]. 中国农机化学报, 2023, 44(7): 220-228.
[8]	陈林涛, 薛俊祥, 马旭, 刘兆祥, 杨文涛, 廖建旺. 智能双充种型孔滚筒杂交稻育秧播种器改进设计与试验[J]. 中国农机化学报, 2023, 44(6): 8-17.
[9]	周显沁, 韩震宇, 刘成源. 基于改进卷积神经网络与图像处理的蚕茧识别方法[J]. 中国农机化学报, 2023, 44(5): 100-106.
[10]	王磊, 袁英, 高玲. 基于改进多元宇宙算法的番茄病害图像识别[J]. 中国农机化学报, 2023, 44(5): 176-181.
[11]	王建翠, 惠巧娟, 吴立国. 基于多尺度注意力和深度可分离卷积的农田杂草检测[J]. 中国农机化学报, 2023, 44(5): 182-187.
[12]	付豪, , 赵学观, 翟长远, , 郑康, 郑申玉, 王秀. 基于深度学习的杂草识别方法研究进展[J]. 中国农机化学报, 2023, 44(5): 198-207.
[13]	李岩舟, 何艳洲, 覃锋, 钱万强, 吴媚, 乔曦, . 基于卷积神经网络的互花米草识别研究[J]. 中国农机化学报, 2023, 44(4): 159-166.
[14]	何雨霜, 王琢, 王湘平, 肖进, 罗友谊, 张俊峰. 深度学习在农作物病害图像识别中的研究进展[J]. 中国农机化学报, 2023, 44(2): 148-155.
[15]	李子茂, 李嘉晖, 尹帆, 帖军, 吴钱宝, . 基于可形变卷积与SimAM注意力的密集柑橘检测算法[J]. 中国农机化学报, 2023, 44(2): 156-162.