Named entity recognition of veterinary drug text based on character and word fusion and attention mechanism

doi:10.13733/j.jcam.issn.20955553.2025.03.047

Abstract

Abstract:

In view of the characteristics of strong professionalism, strong relevance, obvious local features, and polysemy of the information in the field of veterinary drugs and the problem that the mainstream named entity recognition model does not make full use of vocabulary information, a named entity recognition model of veterinary drug text based on character and word fusion and the attention mechanism is proposed. Firstly, the character vector obtained by the pre-training model BERT and the word vector obtained by Word2vec are fused. Secondly, it utilizes a bidirectional long short-term memory network （BiLSTM） to capture overall contextual information and employs a multi-head self-attention mechanism （MHA） to extract local features from sequences. Lastly, a conditional random field （CRF） is utilized to determine the optimal sequence of labels for named entity recognition. Multiple sets of experiments on the veterinary drug text dataset show that the recognition precision, recall rate, and F1—score of the model are 94.73%, 95.29%, and 95.01%, respectively. The performance of the model is better than the comparison model.

Key words: veterinary drug text, named entity recognition, character and word fusion, multi-head self-attention mechanism

摘要：

针对兽药领域信息专业性强、关联性强、局部特征明显和一词多义的特点，以及主流的命名实体识别模型未充分利用词汇信息的问题，提出一种基于字词融合和注意力机制的兽药文本命名实体识别模型。首先，将BERT预训练模型得到的字向量和Word2vec得到的词向量融合。然后，在双向长短期记忆网络中提取全局上下文特征的基础上加入多头自注意力机制挖掘序列的局部特征。最后，通过条件随机场获得最佳标签序列来完成实体识别任务。在兽药文本数据集上进行多组对比试验，结果表明，该模型识别的准确率、召回率和F1值分别为94.73%、95.29%和95.01%，性能均优于对比模型。

关键词: 兽药文本, 命名实体识别, 字词融合, 多头自注意力机制

CLC Number:

TP391

Yan Shijun, Zhu Hongmei, Wang Yatong, Zhang Liang. Named entity recognition of veterinary drug text based on character and word fusion and attention mechanism[J]. Journal of Chinese Agricultural Mechanization, 2025, 46(3): 336-342.

颜士军, 朱红梅, 王雅童, 张亮. 基于字词融合和注意力机制的兽药文本命名实体识别#br#[J]. 中国农机化学报, 2025, 46(3): 336-342.

References

［1］　Liu J, Yang Y, He H. Multi-level semantic representation enhancement network for relationship extraction ［J］. Neurocomputing, 2020, 403: 282-293.

［2］　张善文, 王振, 王祖良. 结合知识图谱与双向长短时记忆网络的小麦条锈病预测［J］. 农业工程学报, 2020, 36（12）: 172-178.

Zhang Shanwen, Wang Zhen, Wang Zuliang.Prediction of wheat stripe rust disease by combining knowledge graph and bidirectional long short term memory network ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36（12）: 172-178.

［3］　Do P, Phan T H V. Developing a BERT based triple classification model using knowledge graph embedding for question answering system ［J］. Applied Intelligence, 2022, 52（1）: 636-651.

［4］　郑丽敏, 齐珊珊, 田立军, 等. 面向食品安全事件新闻文本的实体关系抽取研究［J］. 农业机械学报, 2020, 51（7）: 244-253.

Zheng Limin, Qi Shanshan, Tian Lijun, et al. Entity relationextraction of news texts for food safety events ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51（7）: 244-253.
［5］　Malarkodi C S, Lex E, Devi S L. Named entity recognition for the agricultural domain ［J］. Research in Computing Science, 2016, 117（1）: 121-132.
［6］　李想, 魏小红, 贾璐, 等. 基于条件随机场的农作物病虫害及农药命名实体识别［J］. 农业机械学报, 2017, 48（S1）: 178-185.

Li Xiang, Wei Xiaohong, Jia Lu, et al. Recognition ofcrops, diseases and pesticides named entities in Chinese based on conditional random fields ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48（S1）: 178-185.
［7］　Hochreiter S, Schmidhuber J. Long short-term memory ［J］.Neural Computation, 1997, 9（8）: 1735-1780.
［8］　Huang Z, Xu W, Yu K. Bidirectional LSTM—CRF models for sequence tagging ［J］. Computer Science, 2015, 4（1）: 1508-1519.
［9］　Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding ［J］. arXiv preprint arXiv:1810.04805, 2018.
［10］　陈剑, 何涛, 闻英友, 等. 基于BERT模型的司法文书实体识别方法［J］. 东北大学学报（自然科学版）, 2020, 41（10）: 1382-1387.

Chen Jian, He Tao, Wen Yingyou, et al. Entity recognition method for judicial documents based on BERT model ［J］. Journal of Northeastern University （Natural Science）, 2020, 41（10）: 1382-1387.
［11］　Strubell E, Verga P, Belanger D, et al. Fast and accurate entity recognition with iterated dilated convolutions ［C］. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017: 2670-2680.
［12］　李妮, 关焕梅, 杨飘, 等. 基于BERT—IDCNN—CRF的中文命名实体识别方法［J］. 山东大学学报（理学版）, 2020, 55（1）: 102-109.

Li Ni, Guan Huanmei, Yang Piao, et al. BERT—IDCNN—CRF for named entity recognition in Chinese ［J］. Journal of Shandong University （Natural Science）, 2020, 55（1）: 102-109.

［13］　Zhang Y, Yang J. Chinese NER using lattice LSTM ［C］.Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1554-1564.
［14］　李书琴, 张明美, 刘斌. 融合字词语义信息的猕猴桃种植领域命名实体识别研究［J］. 农业机械学报, 2022, 53（12）: 323-331.
Li Shuqin, Zhang Mingmei, Liu Fu. Kiwifruit planting entity recognition based on character and word information fusion ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53（12）: 323-331.
［15］　Li M Y, Kong F. Combined self-attention mechanism for named entity recognition in social media ［J］. Journal of Tsinghua University （Science and Technology）, 2019, 59（6）: 461-467.
［16］　刘歆宁. 融合多特征及协同注意力的医学命名实体识别［J］. 计算机工程与应用, 2024, 60（6）: 188-198.

Liu Xinning. Medical named entity recognition based on multi-feature and co-attention ［J］. Computer Engineering and Applications, 2024, 60（6）: 188-198.
［17］　Guo X, Zhou H, Su J, et al.Chinese agricultural diseases and pests named entity recognition with multi-scale local context features and self-attention mechanism ［J］. Computers and Electronics in Agriculture, 2020, 179（5）: 105830.
［18］　罗熹, 夏先运, 安莹,等. 结合多头自注意力机制与BiLSTM—CRF的中文临床实体识别［J］. 湖南大学学报（自然科学版）, 2021, 48（4）: 45-55.

Luo Xi, Xia Xianyun, An Ying, et al. Chinese CNER combined with multi-head self-attention and BiLSTM—CRF ［J］. Journal of Hunan University （Natural Sciences）, 2021, 48（4）: 45-55.
［19］　王雅童. 基于知识图谱的兽药知识问答系统研究与实现［D］. 泰安: 山东农业大学, 2022.

Wang Yatong. Research and implementation of veterinary drug knowledge question answering system based on knowledge graph ［D］. Tai'an: Shandong Agricultural University, 2022.

[1]	Li Xianwang, Liu Saihu, Huang Zhongxiang, Zhang Xiadong. Named entity recognition of grain harvesting machinery maintenance knowledge based on XLNet—BiLSTM—AFF—CRF [J]. Journal of Chinese Agricultural Mechanization, 2025, 46(2): 319-325.
[2]	Yang Ning, Qian Ye, , Chen Jian. Research on named entity recognition for combine harvester fault domain [J]. Journal of Chinese Agricultural Mechanization, 2024, 45(8): 338-343.