Image-based recognition of parasitoid wasps using advanced neural networks

Hossein Shirali; Jeremy Hübner; Robin Both; Michael Raupach; Markus Reischl; Stefan Schmidt; Christian Pylatiuk

doi:10.1071/IS24011

RESEARCH ARTICLE (Open Access)

Contents Vol 38(6)

Image-based recognition of parasitoid wasps using advanced neural networks

Hossein Shirali

^A ^* , Jeremy Hübner

^B ^* , Robin Both ^A , Michael Raupach

^B , Markus Reischl

^A , Stefan Schmidt

^C and Christian Pylatiuk

^A

+ Author Affiliations

- Author Affiliations

^A Institute for Automation and Applied Informatics (IAI), Karlsruhe Institute of Technology (KIT), D-76149 Karlsruhe, Germany.

^B Zoologische Staatssammlung München, Münchhausenstraße 21, D-81247 Munich, Germany.

^C Deceased. Formerly at Zoologische Staatssammlung München, Münchhausenstraße 21, D-81247 Munich, Germany.

^* Correspondence to: hossein.shirali@kit.edu, miomio9237@gmail.com

Handling Editor: Gonzalo Giribet

Invertebrate Systematics 38, IS24011 https://doi.org/10.1071/IS24011

Submitted: 30 January 2024 Accepted: 8 May 2024 Published: 5 June 2024

© 2024 The Author(s) (or their employer(s)). Published by CSIRO Publishing. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)

Abstract

Hymenoptera has some of the highest diversity and number of individuals among insects. Many of these species potentially play key roles as food sources, pest controllers and pollinators. However, little is known about the diversity and biology and ~80% of the species have not yet been described. Classical taxonomy based on morphology is a rather slow process but DNA barcoding has already brought considerable progress in identification. Innovative methods such as image-based identification and automation can further speed up the process. We present a proof of concept for image data recognition of a parasitic wasp family, the Diapriidae (Hymenoptera), obtained as part of the GBOL III project. These tiny (1.2–4.5 mm) wasps were photographed and identified using DNA barcoding to provide a solid ground truth for training a neural network. Taxonomic identification was used down to the genus level. Subsequently, three different neural network architectures were trained, evaluated and optimised. As a result, 11 different genera of diaprids and one mixed group of ‘other Hymenoptera’ can be classified with an average accuracy of 96%. Additionally, the sex of the specimen can be classified automatically with an accuracy of >97%.

Keywords: AI, artificial intelligence, biodiversity, Diapriidae, DNA barcoding, genus classification, Hymenoptera, image-based identification, integrative taxonomy, machine learning, neural network architectures, taxonomic identification.

References

Blagoev G, Hebert P, Adamowicz S, Robinson E (2009) Prospects for using DNA barcoding to identify spiders in species-rich genera. ZooKeys 16, 27-46.
| Crossref | Google Scholar |

Borowiec ML, Dikow RB, Frandsen PB, McKeeken A, Valentini G, White AE (2022) Deep learning as a tool for ecology and evolution. Methods in Ecology and Evolution 13, 1640-1660.
| Crossref | Google Scholar |

Chimeno C, Hausmann A, Schmidt S, Raupach MJ, Doczkal D, Baranov V, Hübner J, Höcherl A, Albrecht R, Jaschhof M, Haszprunar G, Hebert PDN (2022) Peering into the darkness: DNA barcoding reveals surprisingly high diversity of unknown species of Diptera (Insecta) in Germany. Insects 13, 82.
| Crossref | Google Scholar | PubMed |

Chimeno C, Rulik B, Manfrin A, Kalinkat G, Hölker F, Baranov V (2023) Facing the infinity: tackling large samples of challenging Chironomidae (Diptera) with an integrative approach. PeerJ 11, e15336.
| Crossref | Google Scholar | PubMed |

da Silva Puls E, Todescato MV, Carbonera JL (2023) An evaluation of pre-trained models for feature extraction in image classification. arXiv v1, 2310.02037 [Preprint, posted 3 October 2023].
| Crossref | Google Scholar |

De Vos JM, Joppa LN, Gittleman JL, Stephens PR, Pimm SL (2015) Estimating the normal background rate of species extinction. Conservation Biology 29, 452-462.
| Crossref | Google Scholar | PubMed |

Dunn RR, Fitzpatrick MC (2012) Every species is an insect (or nearly so): on insects, climate change, extinction, and the biological unknown. In ‘Saving a Million Species’. (Ed. L Hannah) pp. 217–237. (Island Press and Center for Resource Economics: Washington, DC, USA) 10.5822/978-1-61091-182-5_13

Folliot A, Haupert S, Ducrettet M, Sèbe F, Sueur J (2022) Using acoustics and artificial intelligence to monitor pollination by insects and tree use by woodpeckers. Science of The Total Environment 838, 155883.
| Crossref | Google Scholar | PubMed |

Godfray HCJ, Lewis OT, Memmott J (1999) Studying insect diversity in the tropics. Philosophical Transactions of the Royal Society of London – B. Biological Sciences 354, 1811-1824.
| Crossref | Google Scholar | PubMed |

Goldstein PZ, DeSalle R (2011) Integrating DNA barcode data and taxonomic practice: determination, discovery, and description. BioEssays 33, 135-147.
| Crossref | Google Scholar | PubMed |

Hallmann CA, Sorg M, Jongejans E, Siepel H, Hofland N, Schwan H, Stenmans W, Müller A, Sumser H, Hörren T, Goulson D, de Kroon H (2017) More than 75 percent decline over 27 years in total flying insect biomass in protected areas. PLoS One 12, e0185809.
| Crossref | Google Scholar |

Hartop E, Srivathsan A, Ronquist F, Meier R (2022) Towards large-scale integrative taxonomy (LIT): resolving the data conundrum for dark taxa. Systematic Biology 71, 1404-1422.
| Crossref | Google Scholar |

Hausmann A, Krogmann L, Peters RS, Rduch V, Schmidt S (2020) GBOL III: dark taxa. iBOL Barcode Bulletin 10, 2-4.
| Crossref | Google Scholar |

Hebert PDN, Ratnasingham S, De Waard JR (2003) Barcoding animal life: cytochrome c oxidase subunit I divergences among closely related species. Proceedings of the Royal Society of London – B. Biological Sciences 270, S96-S99.
| Crossref | Google Scholar | PubMed |

Hübner J, Shirali H (2024) DiapriidaeGenusImageDataset. Zenodo v2, 22 April 2024 [Data set].
| Crossref | Google Scholar |

Hübner J, Chemyreva VG, Notton D (2023) Taxonomic and nomenclatural notes on Geodiapria longiceps Kieffer, 1911 (Hymenoptera, Diapriidae) and synonymy of the genus Geodiapria Kieffer, 1910. ZooKeys 1183, 1-11.
| Crossref | Google Scholar | PubMed |

Johnson N (1992) Catalog of world species of Proctotrupoidea, exclusive of Platygastridae (Hymenoptera). Memoirs of the American Entomological institute 51, 1-825.
| Crossref | Google Scholar |

Li Z, Gu T, Li B, Xu W, He X, Hui X (2022) ConvNeXt-based fine-grained image classification and bilinear attention mechanism model. Applied Sciences 12, 9016.
| Crossref | Google Scholar |

Liu S, Zeng Z, Ren T, Li F, Zhang H, Yang J, Li C, Yang J, Su H, Zhu J (2023) Grounding DINO: marrying dino with grounded pre-training for open-set object detection. arXiv v4, 2303.05499 [Preprint, posted 23 March 2023] .
| Crossref | Google Scholar |

Mora C, Tittensor DP, Adl S, Simpson AGB, Worm B (2011) How many species are there on Earth and in the ocean? PLoS Biology 9, e1001127.
| Crossref | Google Scholar |

Morinière J, Balke M, Doczkal D, Geiger MF, Hardulak LA, Haszprunar G, Hausmann A, Hendrich L, Regalado L, Rulik B, Schmidt S, Wägele JW, Hebert PDN (2019) A DNA barcode library for 5,200 German flies and midges (Insecta: Diptera) and its implications for metabarcoding‐based biomonitoring. Molecular Ecology Resources 19, 900-928.
| Crossref | Google Scholar | PubMed |

Nixon GEJ (1980) ‘Diapriidae (Diapriinae): Hymenoptera, Proctotrupoidea. Handbooks for the Identification of British Insects, Volume VIII, Part 3(di).’ (Ed. G Fitton) (Royal Entomological Society of London: London, UK)

Padial JM, Miralles A, De La Riva I, Vences M (2010) The integrative future of taxonomy. Frontiers in Zoology 7, 16.
| Crossref | Google Scholar | PubMed |

Panzer GWF (1801) ‘Faunae insectorum germanicae initia oder Deutschlands Insecten.’ (Felseckersche Buchhandlung: Nürnberg, Holy Roman Empire)

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay É (2011) Scikit-learn: machine learning in Python. Journal of Machine Learning Research 12, 2825-2830 Available at http://jmlr.org/papers/v12/pedregosa11a.html.
| Google Scholar |

Peng Z, Dong L, Bao H, Ye Q, Wei F (2022) BEiTv2: masked image modeling with vector-quantized visual tokenizers. arXiv v2, 2208.06366 [Preprint, posted 3 October 2022].
| Crossref | Google Scholar |

Rainford JL, Hofreiter M, Mayhew PJ (2016) Phylogenetic analyses suggest that diversification and body size evolution are independent in insects. BMC Evolutionary Biology 16, 8.
| Crossref | Google Scholar | PubMed |

Rossi Stacconi MV, Grassi A, Ioriatti C, Anfora G (2019) Augmentative releases of Trichopria drosophilae for the suppression of early season Drosophila suzukii populations. BioControl 64, 9-19.
| Crossref | Google Scholar |

Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 211-252.
| Crossref | Google Scholar |

Say T (1836) Descriptions of new species of North American Hymenoptera, and observations on some already described. Boston Journal of Natural History 1, 209-305.
| Google Scholar |

Schlick-Steiner BC, Steiner FM, Seifert B, Stauffer C, Christian E, Crozier RH (2010) Integrative taxonomy: a multisource approach to exploring biodiversity. Annual Review of Entomology 55, 421-438.
| Crossref | Google Scholar | PubMed |

Shirali H, Hübner J, Both R, Raupach M, Schmidt S, Pylatiuk C (2024) Image-based recognition of parasitoid wasps using advanced neural networks. bioRxiv 2024, 2024.01.01.573817 [Preprint, posted 2 January 2024].
| Crossref | Google Scholar |

Stork NE (2018) How many species of insects and other terrestrial arthropods are there on Earth? Annual Review of Entomology 63, 31-45.
| Crossref | Google Scholar | PubMed |

Toscano-Miranda R, Toro M, Aguilar J, Caro M, Marulanda A, Trebilcok A (2022) Artificial-intelligence and sensing techniques for the management of insect pests and diseases in cotton: a systematic literature review. The Journal of Agricultural Science 160, 16-31.
| Crossref | Google Scholar |

Wührl L, Pylatiuk C, Giersch M, Lapp F, von Rintelen T, Balke M, Schmidt S, Cerretti P, Meier R (2022) DiversityScanner: robotic handling of small invertebrates with machine learning methods. Molecular Ecology Resources 22, 1626-1638.
| Crossref | Google Scholar | PubMed |

Wührl L, Rettenberger L, Meier R, Hartop E, Graf J, Pylatiuk C (2024) Entomoscope: an open-source photomicroscope for biodiversity discovery. IEEE Access 12, 11785-11794.
| Crossref | Google Scholar |

Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In ‘Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)’, 27–30 June 2016, Las Vegas, NV, USA. pp. 2921–2929. (IEEE) 10.1109/CVPR.2016.319