ICCK Transactions on Machine Intelligence Home About Editors Current Issue
Tomato Multimodal Knowledge Graph Construction and Dynamic Updating
Research Article  ·  Published: 22 October 2026
Issue cover
ICCK Transactions on Machine Intelligence
Volume 2, Issue 4, 2026: 224-235
Research Article Open Access

Tomato Multimodal Knowledge Graph Construction and Dynamic Updating

by
Ziming Gao Ziming Gao 1,2,3,4,5 ORCID
Huaji Zhu Huaji Zhu 1,2,3,4,5 * ORCID
Yusen Yang Yusen Yang 2,3,4,5
1 College of Computer Science and Technology, Inner Mongolia Minzu University, Tongliao 028000, China
2 National Engineering Research Center for Information Technology in Agriculture, Beijing 100097, China
3 Information Technology Research Center, Beijing Academy of Agriculture and Forestry Sciences, Beijing 100097, China
4 Key Laboratory of Digital Village Technology, Ministry of Agriculture and Rural Affairs, Beijing 100097, China
5 Beijing Research Institute of Agricultural Artificial Intelligence and Robotics, Beijing 100097, China
* Corresponding Author: Huaji Zhu, [email protected]
Volume 2, Issue 4
Volume 2, Issue 4 2026
Received: 11 August 2026 Revised: 01 October 2026 Accepted: 20 October 2026 Published: 22 October 2026 CrossMark
Download PDF 6.79 MB Cite Metrics
DOI: 10.62762/TMI.2026.572797
ARK: ark:/57805/tmi.2026.572797
You have full access to this open access article · CC BY 4.0 License

Article Information

Published in ICCK Transactions on Machine Intelligence
Volume/Issue Volume 2, Issue 4, 2026
Pages 224-235

Abstract

Tomato disease knowledge is scattered across agronomic texts and leaf images from heterogeneous sources, and static knowledge graphs that write extraction outputs directly to a database cannot separate model candidates from currently valid facts. We construct a multimodal knowledge graph for tomato diseases together with an evidence-gated dynamic update workflow. On the text side, a Chinese-RoBERTa-wwm-ext encoder with GlobalPointer identifies entity spans, and a directed relation pointer network extracts relations. On the image side, Chinese-CLIP fine-tuned with supervised contrastive learning aligns disease images with disease-text prototypes. On the graph side, stable entity and fact identifiers, source evidence and stateful writes support traceable maintenance, and a disease-centred interface visualises the graph. Named-entity recognition (NER) and relation extraction (RE) achieved F1 scores of 0.7326 and 0.7042. In a source-group-disjoint alignment test, image-to-text Recall@1 was 0.8987, text-to-image Recall@1 was 0.9500, and prototype classification reached an Accuracy of 0.9295 and a Macro-F1 of 0.9317. For dynamic updating, evidence gating achieved Precision/Recall/F1 of 0.9643/0.9000/0.9310, and the false-admission rate fell from 0.2308 (direct writing) to 0.0357. Checking relation structure, local semantics and source evidence before admission therefore removed most erroneous facts, while every admitted fact remained linked to its evidence and could be replayed when models or rules change.

Graphical Abstract

Tomato Multimodal Knowledge Graph Construction and Dynamic Updating

Keywords

multimodal knowledge graph named-entity recognition relation extraction image-text alignment dynamic updating

Data Availability Statement

Data will be made available on request.

Funding

This work was supported by the National Major Agricultural Science and Technology Project under Grant 0304.

Conflicts of Interest

The authors declare no conflicts of interest.

AI Use Statement

The authors declare that no generative AI was used in the preparation of this manuscript.

Ethical Approval and Consent to Participate

Not applicable. This study did not involve human participants or animals; all images are photographs of plant leaves.

References

  1. Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G. D., Gutierrez, C., ... & Zimmermann, A. (2021). Knowledge graphs. ACM Computing Surveys (Csur), 54(4), 1-37.
    [CrossRef] [Google Scholar]
  2. Wang, K., Miao, Y., Wang, X., Li, Y., Li, F., & Song, H. (2024). Research on the construction of a knowledge graph for tomato leaf pests and diseases based on the named entity recognition model. Frontiers in Plant Science, 15, 1482275.
    [CrossRef] [Google Scholar]
  3. Yan, R., An, P., Meng, X., Li, Y., Li, D., Xu, F., & Dang, D. (2025). A knowledge graph for crop diseases and pests in China. Scientific Data, 12(1), 222.
    [CrossRef] [Google Scholar]
  4. Lv, B., Wu, H., Chen, W., Chen, C., Miao, Y., & Zhao, C. (2024). VEG-MMKG: Multimodal knowledge graph construction for vegetables based on pre-trained model extraction. Computers and Electronics in Agriculture, 226, 109398.
    [CrossRef] [Google Scholar]
  5. Zhong, L., Wu, J., Li, Q., Peng, H., & Wu, X. (2023). A comprehensive survey on automatic knowledge graph construction. ACM Computing Surveys, 56(4), 1-62.
    [CrossRef] [Google Scholar]
  6. Zhao, X., Chen, B., Ji, M., Wang, X., Yan, Y., Zhang, J., ... & Lv, C. (2024). Implementation of large language models and agricultural knowledge graphs for efficient plant disease detection. Agriculture, 14(8), 1359.
    [CrossRef] [Google Scholar]
  7. Lu, Y., Lu, X., Zheng, L., Sun, M., Chen, S., Chen, B., ... & Lv, C. (2024). Application of multimodal transformer model in intelligent agricultural disease detection and question-answering systems. Plants, 13(7), 972.
    [CrossRef] [Google Scholar]
  8. Zhang, W., Wang, C., Wu, H., Zhao, C., Teng, G., Huang, S., & Liu, Z. (2022). Research on the Chinese named-entity–relation-extraction method for crop diseases based on BERT. Agronomy, 12(9), 2130.
    [CrossRef] [Google Scholar]
  9. Liang, J., Li, D., Lin, Y., Wu, S., & Huang, Z. (2023). Named entity recognition of Chinese crop diseases and pests based on RoBERTa-wwm with adversarial training. Agronomy, 13(3), 941.
    [CrossRef] [Google Scholar]
  10. Zhang, D., Zheng, G., Liu, H., Ma, X., & Xi, L. (2023). AWdpCNER: Automated Wdp Chinese named entity recognition from wheat diseases and pests text. Agriculture, 13(6), 1220.
    [CrossRef] [Google Scholar]
  11. Tang, W., Wen, X., & Hu, Z. (2024). Named entity recognition for crop diseases and pests based on gated fusion unit and manhattan attention. Agriculture, 14(9), 1565.
    [CrossRef] [Google Scholar]
  12. Wang, P., Zhang, C., Wang, D., Zhang, S., Wang, J., Wang, X., & Huang, L. (2023). Relation extraction for knowledge graph generation in the agriculture domain: A case study on soybean pests and disease. Applied engineering in agriculture, 39(2), 215-224.
    [CrossRef] [Google Scholar]
  13. Veena, G., Gupta, D., & Kanjirangat, V. (2023). Semi-supervised bootstrapped syntax-semantics-based approach for agriculture relation extraction for knowledge graph creation and reasoning. IEEE Access, 11, 138375-138398.
    [CrossRef] [Google Scholar]
  14. Shang, Y. M., Huang, H., & Mao, X. (2022, June). Onerel: Joint entity and relation extraction with one module in one step. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 10, pp. 11285-11293).
    [CrossRef] [Google Scholar]
  15. Liu, B., Tao, J., Chen, W., Zhang, Y., Chen, M., He, L., & Tang, D. (2024). Integration of relation filtering and multi-task learning in globalpointer for entity and relation extraction. Applied Sciences, 14(15), 6832.
    [CrossRef] [Google Scholar]
  16. Cui, Y., Che, W., Liu, T., Qin, B., Wang, S., & Hu, G. (2020, November). Revisiting pre-trained models for Chinese natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2020 (pp. 657-668).
    [CrossRef] [Google Scholar]
  17. Shoaib, M., Shah, B., EI-Sappagh, S., Ali, A., Ullah, A., Alenezi, F., ... & Ali, F. (2023). An advanced deep learning models-based plant disease detection: A review of recent research. Frontiers in plant science, 14, 1158933.
    [CrossRef] [Google Scholar]
  18. Ngo, Q. H., Kechadi, T., & Le-Khac, N. A. (2022). Knowledge representation in digital agriculture: A step towards standardised model. Computers and Electronics in Agriculture, 199, 107127.
    [CrossRef] [Google Scholar]
  19. Jelali, M. (2024). Deep learning networks-based tomato disease and pest detection: A first review of research studies using real field datasets. Frontiers in Plant Science, 15, 1493322.
    [CrossRef] [Google Scholar]
  20. Xu, M., Park, J. E., Lee, J., Yang, J., & Yoon, S. (2024). Plant disease recognition datasets in the age of deep learning: challenges and opportunities. Frontiers in Plant Science, 15, 1452551.
    [CrossRef] [Google Scholar]
  21. Zhang, N., Wu, H., Zhu, H., Deng, Y., & Han, X. (2022). Tomato disease classification and identification method based on multimodal fusion deep learning. Agriculture, 12(12), 2014.
    [CrossRef] [Google Scholar]
  22. Yang, A., Pan, J., Lin, J., Men, R., Zhang, Y., Zhou, J., & Zhou, C. (2022). Chinese clip: Contrastive vision-language pretraining in chinese. arXiv preprint arXiv:2211.01335.
    [CrossRef] [Google Scholar]
  23. Zhou, Y., Yan, H., Ding, K., Cai, T., & Zhang, Y. (2024). Few-shot image classification of crop diseases based on vision–language models. Sensors, 24(18), 6109.
    [CrossRef] [Google Scholar]
  24. Liu, W., Wu, G., Wang, H., & Ren, F. (2025). Cross-modal data fusion via vision-language model for crop disease recognition. Sensors, 25(13), 4096.
    [CrossRef] [Google Scholar]
  25. Wang, Y., Wang, F., Chen, W., Lv, B., Liu, M., Kong, X., ... & Pan, Z. (2025). A large language model for multimodal identification of crop diseases and pests. Scientific Reports, 15(1), 21959.
    [CrossRef] [Google Scholar]
  26. Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2021). A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2), 494-514.
    [CrossRef] [Google Scholar]
  27. Hofer, M., Obraczka, D., Saeedi, A., Köpcke, H., & Rahm, E. (2024). Construction of knowledge graphs: Current state and challenges. Information, 15(8), 509.
    [CrossRef] [Google Scholar]
  28. Cai, B., Xiang, Y., Gao, L., Zhang, H., Li, Y., & Li, J. (2023). Temporal knowledge graph completion: A survey. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (pp. 6545-6553).
    [CrossRef] [Google Scholar]
  29. Sikos, L. F., & Philp, D. (2020). Provenance-aware knowledge representation: A survey of data models and contextualized knowledge graphs. Data Science and Engineering, 5(3), 293-316.
    [CrossRef] [Google Scholar]
  30. Ayadi, N. Y., Bernard, S., Bossy, R., Courtin, M., Happi, B. G. H., Larmande, P., ... & Faron, C. (2024). A unified approach to publish semantic annotations of agricultural documents as knowledge graphs. Smart Agricultural Technology, 8, 100484.
    [CrossRef] [Google Scholar]
  31. Hughes, D., & Salathé, M. (2015). An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv preprint arXiv:1511.08060.
    [CrossRef] [Google Scholar]
  32. Su, J., Murtadha, A., Pan, S., Hou, J., Sun, J., Huang, W., ... & Liu, Y. (2022). Global pointer: Novel efficient span-based approach for named entity recognition. arXiv preprint arXiv:2208.03054.
    [CrossRef] [Google Scholar]
  33. Wu, L., Li, J., Wang, Y., Meng, Q., Qin, T., Chen, W., ... & Liu, T. Y. (2021). R-drop: Regularized dropout for neural networks. Advances in neural information processing systems, 34, 10890-10905.
    [Google Scholar]
  34. Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., ... & Krishnan, D. (2020). Supervised contrastive learning. Advances in neural information processing systems, 33, 18661-18673.
    [Google Scholar]

Cite This Article

APA Style
Gao, Z., Zhu, H., & Yang, Y. (2026). Tomato Multimodal Knowledge Graph Construction and Dynamic Updating. ICCK Transactions on Machine Intelligence, 2(4), 224-235. https://doi.org/10.62762/TMI.2026.572797
Export Citation
RIS Format
Compatible with EndNote, Zotero, Mendeley, and other reference managers
TY  - JOUR
AU  - Gao, Ziming
AU  - Zhu, Huaji
AU  - Yang, Yusen
PY  - 2026
DA  - 2026/10/22
TI  - Tomato Multimodal Knowledge Graph Construction and Dynamic Updating
JO  - ICCK Transactions on Machine Intelligence
T2  - ICCK Transactions on Machine Intelligence
JF  - ICCK Transactions on Machine Intelligence
VL  - 2
IS  - 4
SP  - 224
EP  - 235
DO  - 10.62762/TMI.2026.572797
UR  - https://www.icck.org/article/abs/TMI.2026.572797
KW  - multimodal knowledge graph
KW  - named-entity recognition
KW  - relation extraction
KW  - image-text alignment
KW  - dynamic updating
AB  - Tomato disease knowledge is scattered across agronomic texts and leaf images from heterogeneous sources, and static knowledge graphs that write extraction outputs directly to a database cannot separate model candidates from currently valid facts. We construct a multimodal knowledge graph for tomato diseases together with an evidence-gated dynamic update workflow. On the text side, a Chinese-RoBERTa-wwm-ext encoder with GlobalPointer identifies entity spans, and a directed relation pointer network extracts relations. On the image side, Chinese-CLIP fine-tuned with supervised contrastive learning aligns disease images with disease-text prototypes. On the graph side, stable entity and fact identifiers, source evidence and stateful writes support traceable maintenance, and a disease-centred interface visualises the graph. Named-entity recognition (NER) and relation extraction (RE) achieved F1 scores of 0.7326 and 0.7042. In a source-group-disjoint alignment test, image-to-text Recall@1 was 0.8987, text-to-image Recall@1 was 0.9500, and prototype classification reached an Accuracy of 0.9295 and a Macro-F1 of 0.9317. For dynamic updating, evidence gating achieved Precision/Recall/F1 of 0.9643/0.9000/0.9310, and the false-admission rate fell from 0.2308 (direct writing) to 0.0357. Checking relation structure, local semantics and source evidence before admission therefore removed most erroneous facts, while every admitted fact remained linked to its evidence and could be replayed when models or rules change.
SN  - 3068-7403
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  -
BibTeX Format
Compatible with LaTeX, BibTeX, and other reference managers
@article{Gao2026Tomato,
  author = {Ziming Gao and Huaji Zhu and Yusen Yang},
  title = {Tomato Multimodal Knowledge Graph Construction and Dynamic Updating},
  journal = {ICCK Transactions on Machine Intelligence},
  year = {2026},
  volume = {2},
  number = {4},
  pages = {224-235},
  doi = {10.62762/TMI.2026.572797},
  url = {https://www.icck.org/article/abs/TMI.2026.572797},
  abstract = {Tomato disease knowledge is scattered across agronomic texts and leaf images from heterogeneous sources, and static knowledge graphs that write extraction outputs directly to a database cannot separate model candidates from currently valid facts. We construct a multimodal knowledge graph for tomato diseases together with an evidence-gated dynamic update workflow. On the text side, a Chinese-RoBERTa-wwm-ext encoder with GlobalPointer identifies entity spans, and a directed relation pointer network extracts relations. On the image side, Chinese-CLIP fine-tuned with supervised contrastive learning aligns disease images with disease-text prototypes. On the graph side, stable entity and fact identifiers, source evidence and stateful writes support traceable maintenance, and a disease-centred interface visualises the graph. Named-entity recognition (NER) and relation extraction (RE) achieved F1 scores of 0.7326 and 0.7042. In a source-group-disjoint alignment test, image-to-text Recall@1 was 0.8987, text-to-image Recall@1 was 0.9500, and prototype classification reached an Accuracy of 0.9295 and a Macro-F1 of 0.9317. For dynamic updating, evidence gating achieved Precision/Recall/F1 of 0.9643/0.9000/0.9310, and the false-admission rate fell from 0.2308 (direct writing) to 0.0357. Checking relation structure, local semantics and source evidence before admission therefore removed most erroneous facts, while every admitted fact remained linked to its evidence and could be replayed when models or rules change.},
  keywords = {multimodal knowledge graph, named-entity recognition, relation extraction, image-text alignment, dynamic updating},
  issn = {3068-7403},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations
Crossref
0
Scopus
0
Views
21
PDF Downloads
4

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

CC BY Copyright © 2026 by the Author(s). Published by Institute of Central Computation and Knowledge. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
ICCK Transactions on Machine Intelligence
ICCK Transactions on Machine Intelligence
ISSN: 3068-7403 (Online)
Portico
Preserved at
Portico

Article QR Code

Article QR Code
Scan to read this article
Submit Manuscript Edit a Special Issue

Popular Articles

Share This Article