基于多模态分层注意力网络的玉米病害叶片识别

doi:10.13733/j.jcam.issn.2095-5553.2025.06.024

中国农机化学报 ›› 2025, Vol. 46 ›› Issue (6): 162-168.DOI: 10.13733/j.jcam.issn.2095-5553.2025.06.024

• 设施农业与植保机械工程 • 上一篇下一篇

基于多模态分层注意力网络的玉米病害叶片识别

商炎亮1，吴凯2，周文3

（1. 许昌电气职业学院，河南许昌， 461000； 2. 郑州大学信息工程学院，郑州市，450001；
3. 中国科学院计算技术研究所计算机体系结构国家重点实验室，北京市，100190）

出版日期:2025-06-15 发布日期:2025-05-22
基金资助:
国家自然科学基金（61972362）

Leaf recognition of corn disease based on multimodal hierarchical attention network

Shang Yanliang1, Wu Kai2, Zhou Wen3

（1. Xuchang Electrical Vocational College, Xuchang, 461000, China; 2. School of Information and Engineering, Zhengzhou University, Zhengzhou, 450001, China; 3. State Key Laboratory of Computer Architecture, Institute of Computing Technology of Chinese Academy of Sciences, Beijing, 100190, China）

Online:2025-06-15 Published:2025-05-22

摘要/Abstract

摘要：

玉米叶片病害会影响叶片光合作用，严重影响玉米籽粒的灌浆，及时准确地检测玉米病害叶片有助于提高玉米的产量和质量,为此提出一种基于多模态分层注意力网络的小样本玉米病害叶片识别方法。首先，利用视觉特征提取网络VGG16将输入的玉米叶片映射到视觉语义空间，并逐层计算支持分支和查询分支间的语义关联。然后，利用文本转换器将玉米叶片文本标签映射到文本语义空间，并利用模态交叉注意力建立视觉与文本之间的上下文语义关联，尽可能地聚焦病害区域。最后，利用掩码平均池化技术生成指导未知玉米叶片病害的泛化原型集。在自建和开源的玉米病害叶片数据集上进行测试。结果表明：所提出模型在自建数据集上可以实现96.08%的识别精度，在开源的Plant Village数据集上可以实现98.11%的识别精度。

关键词: 玉米病害, 叶片识别, 视觉语义, 文本语义, 多模态分层注意力

Abstract:

Corn leaf diseases can affect the photosynthesis of the corn leaves, and seriously affect the filling of corn kernels. Therefore, timely and accurate detection and identification of corn diseased leaves is helpful to improve the yield and quality of corn. To address this problem, a recognition method of corn disease leaf in small sample based on multi‑modal hierarchical attention network is proposed. Firstly, a visual feature extraction network VGG16 is used to map the input corn leaf into the visual semantic space and the semantic correlation between the supporting branches and query branches is calculated layer by layer. Secondly, a text converter is used to map the corn leaf text labels into the text semantic space, and a modality cross‑attention mechanism is employed to establish the contextual semantic correlation between vision and text, focusing on the disease area as much as possible. Finally, a masked average pooling technique is used to generate a generalized prototype set that guides the identification of unknown corn diseased leaves. Through testing on self‑built and open‑source corn disease leaf datasets, the proposed model achieves an recognition accuracy of 96.08% on the self‑built dataset and 98.11% on the open source Plant Village dataset.

Key words: corn disease, leaf recognition, visual semantics, text semantics, multimodal hierarchical attention

中图分类号:

商炎亮, 吴凯, 周文. 基于多模态分层注意力网络的玉米病害叶片识别[J]. 中国农机化学报, 2025, 46(6): 162-168.

Shang Yanliang, Wu Kai, Zhou Wen. Leaf recognition of corn disease based on multimodal hierarchical attention network[J]. Journal of Chinese Agricultural Mechanization, 2025, 46(6): 162-168.

参考文献

[ 1 ] Abdul W, Muskan G, Deepak G, et al. An optimized dense convolutional neural network model for disease recognition and classification in corn leaf [J]. Computers and Electronics in Agriculture, 2020, 175: 105456.

[ 2 ] Sumita M, Rishabh S, Diksha R. Deep convolutional neural network based detection system for real‑time corn plant disease recognition [J]. Procedia Computer Science, 2020, 167: 2003-2010.

[ 3 ] 张梓婷, 韩金玉, 张东辉, 等. 基于颜色矩的土豆、玉米、苹果叶片病害异常检测[J]. 浙江农业学报, 2022, 34(10): 2230-2239.

[ 4 ] Zeng W, Li H, Hu G, et al. Lightweight dense‑scale network (LDSNet) for corn leaf disease identification [J]. Computers and Electronics in Agriculture, 2022, 197: 106943.

[ 5 ] Zhao Y, Sun C, Xu X, et al. RIC—Net: A plant disease classification model based on the fusion of inception and residual structure and embedded attention mechanism [J]. Computers and Electronics in Agriculture, 2022, 193: 106644.

[ 6 ] 黄英来, 艾昕. 改进残差网络在玉米叶片病害图像的分类研究[J]. 计算机工程与应用, 2021, 57(23): 178-184.

[ 7 ] 王大庆, 禄琳, 于兴龙, 等. 基于深度迁移学习的EfficientNet玉米叶部病害识别[J]. 东北农业大学学报, 2023, 54(5): 66-76.

Wang Daqing, Lu Lin, Yu Xinglong, et al. Maize leaf diseases identification using EfficientNet based on deep‑transfer learning [J]. Journal of Northeast Agricultural University, 2023, 54(5): 66-76.

[ 8 ] 熊梦园, 詹炜, 桂连友, 等. 基于ResNet模型的玉米叶片病害检测与识别[J]. 江苏农业科学, 2023, 51(8): 164-170.

[ 9 ] 惠巧娟, 孙婕. 基于多尺度特征度量元学习的玉米叶片病害识别模型研究[J]. 江苏农业科学, 2023, 51(9): 199-206.

[10] 曹藤宝, 张欣, 陈孝玉龙, 等. 融合空间注意力机制和DenseNet的玉米病害分类方法[J]. 无线电工程, 2022, 52(10): 1710-1717.

[11] 桂江生, 费婧怡, 傅霞萍. 三维小样本元学习模型的大豆食心虫虫害高光谱检测[J]. 光谱学与光谱分析, 2021, 41(7): 2171-2174.

[12] Xu W, Li W, Wang L, et al. Enhancing corn pest and disease recognition through deep learning: A comprehensive analysis [J]. Agronomy, 2023, 13(9): 2242.

[13] Gangadharan V, Vani K, Deepa G. AGRONER: An unsupervised agriculture named entity recognition using weighted distributional semantic model [J]. Expert Systems with Applications, 2023, 229: 120440.

[14] 鲍文霞, 黄雪峰, 胡根生, 等. 基于改进卷积神经网络模型的玉米叶部病害识别[J]. 农业工程学报, 2021, 37(6): 160-167.

Bao Wenxia, Huang Xuefeng, Hu Gensheng, et al. Identification of maize leaf diseases using improved convolutional neural network [J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(6): 160-167.

[15] Bi C, Xu S, Hu N, et al. Identification method of corn leaf disease based on improved MobileNet v3 model [J]. Agronomy, 2023, 13(2): 300.

[1]	李思进, , 李金阳, , 张伟, . 基于改进YOLOv5s的大豆冠层叶片识别方法研究[J]. 中国农机化学报, 2025, 46(5): 100-105.
[2]	牛玉霞, , 孙宙红, 任伟, , 陈林琳, , 陈莉莉, . 基于分层特征对齐网络的小样本马铃薯病害叶片检测[J]. 中国农机化学报, 2024, 45(2): 250-258.
[3]	张净, 张康, 刘晓梅, 杨宁. 基于物联网的远程温室视觉监控系统设计与实现[J]. 中国农机化学报, 2023, 44(1): 100-107.

基于多模态分层注意力网络的玉米病害叶片识别

Leaf recognition of corn disease based on multimodal hierarchical attention network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价