Abstract
We apply deep metric learning for the first time to the problem of classifying planktic foraminifer shells on microscopic images. This species recognition task is an important information source and scientific pillar for reconstructing past climates. All foraminifer CNN recognition pipelines in the literature produce black-box classifiers that lack visualisation options for human experts and cannot be applied to open set problems. Here, we benchmark metric learning against these pipelines, produce the first scientific visualisation of the phenotypic planktic foraminifer morphology space, and demonstrate that metric learning can be used to cluster species unseen during training. We show that metric learning outperforms all published CNN-based state-of-the-art benchmarks in this domain. We evaluate our approach on the 34,640 expert-annotated images of the Endless Forams public library of 35 modern planktic foramini-fera species. Our results on this data show leading \(92\%\) accuracy (at 0.84 F1-score) in reproducing expert labels on withheld test data, and \(66.5\%\) accuracy (at 0.70 F1-score) when clustering species never encountered in training. We conclude that metric learning is highly effective for this domain and serves as an important tool towards expert-in-the-loop automation of microfossil identification. Key code, network weights, and data splits are published with this paper for full reproducibility.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Tail classes 1, 5, 9, 14, 22, 23, 26, 29, 33, and 34 were chosen as our open set to have maximum specimen counts available during training.
References
Al-Sabouni, N., Fenton, I., Telford, R., Kučera, M.: Reproducibility of species recognition in modern planktonic foraminifera and its implications for analyses of community structure. J. Micropalaeontol. 37, 519–534 (2018)
Andrew, W., Gao, J., Mullan, S., Campbell, N., Dowsey, A.W., Burghardt, T.: Visual identification of individual Holstein-Friesian cattle via deep metric learning. Comput. Electron. Agric. 185, 106133 (2021)
Aurahs, R., Treis, Y., Darling, K., Kucera, M.: A revised taxonomic and phylogenetic concept for the planktonic foraminifer species Globigerinoides Ruber based on molecular and morphometric evidence. Mar. Micropaleontol. 79, 1–14 (2011)
Balfoort, H., Snoek, J., Smiths, J., Breedveld, L., Hofstraat, J., Ringelberg, J.: Automatic identification of Algae: neural network analysis of flow cytometric data. J. Plankton Res. 14, 575–589 (1992)
Beaufort, L., Dollfus, D.: Automatic recognition of coccoliths by dynamical neural networks. Mar. Micropaleontol. 51, 57–73 (2004)
Beaufort, L., et al.: Sensitivity of coccolithophores to carbonate chemistry and ocean acidification. Nature 476, 80–83 (2011)
Bown, P., Huber, B., Wade, B., Young, J.: pforams@mikrotax - introduction (2020). https://www.mikrotax.org/pforams/. Accessed 01 May 2021
Bozinovski, S., Fulgosi, A.: The use of artificial neural networks to classify primate vocalizations: a pilot study on black Lemurs. Am. J. Primatol. 1098–2345 (2009)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Elder, L.E., Hsiang, A.Y., Nelson, K., Strotz, L.C., Kahanamoku, S.S., Hull, P.M.: Sixty-one thousand recent planktonic foraminifera from the Atlantic Ocean. Sci. Data 5 (2018)
Franz, N.: On the lack of good scientific reasons for the growing phylogeny/classification gap. Cladistics 21, 495–500 (2005)
Gandhi, G., Srivastava, R.: Review paper: A comparative study on partitioning techniques of clustering algorithms. International Journal of Computer Applications 87 (01 2014)
Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 1735–1742 (2006)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
Hemleben, C., Spindler, M., Anderson, O.: Modern Planktonic Foraminifera, p. 363. Springer, Cham (1989). https://doi.org/10.1007/978-1-4612-3544-6
Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification (2017)
Hodan, T., Haluza, P., Obdrzalek, S., Matas, J., Lourakis, M., Zabulis, X.: T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects. In: Winter Conference on Applications of Computer Vision (WACV). IEEE, 880–888 (2017)
Hodges, L.: Nonparametric discrimination: Consistency properties. USAF School of Aviation Medicine (1951)
Hsiang, A.Y., et al.: Endless Forams: \(>\) 34,000 modern planktonic foraminiferal images for taxonomic training and automated species recognition using convolutional neural networks. Paleoceanography Paleoclimatol. 34, 1157–1177 (2019)
Huber, B., Bijma, J., Darling, K.: Cryptic speciation in the living planktonic foraminifer Globigerinella siphoniphera (d’Orbigny). Paleobiology 23, 33–62 (1997)
Hull., P.M., Hsiang, A.Y.: Endless Forams Most Beautiful (2020). http://endlessforams.org. Accessed 01 May 2021
Johnson, S.C.: Hierarchical clustering schemes. Psychometrika 32, 241–254 (1967)
Karaderi, T.: Visual microfossil identification via deep metric learning (2022). https://github.com/TayfunKaraderi/ICPRAI-2022-Visual-Microfossil-Identification-via-Deep-Metric-Learning. Accessed 19 Mar 2022
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Kucera, M.: Planktonic foraminifera as tracers of past oceanic environments. In: Hillaire-Marcel, C., De Vernal, A. (eds.) pp. 213–262 (2007)
Kühl, H.S., Burghardt, T.: Animal biometrics: quantifying and detecting phenotypic appearance. Trends Ecol. Evol. 28(7), 432–441 (2013)
Lagunes-Fortiz, M., Damen, D., Mayol-Cuevas, W.: Learning discriminative embeddings for object recognition on-the-y. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE, pp. 2932–2938 (2019)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Malmgren, B., Kennett, J.: Biometric analysis of phenotypic variation in recent globigerina bulloides d’Orbigny in the southern Indian ocean. Mar. Micropaleontol. 1, 2–25 (1976)
Marchant, R., Tetard, M., Pratiwi, A., Adebayo, M., de Garidel-Thoron, T.: Automated analysis of foraminifera fossil records by image classification using a convolutional neural network. J. Micropalaeontol. 39(2), 183–202 (2020)
Masullo, A., Burghardt, T., Damen, D., Perrett, T., Mirmehdi, M.: Who goes there? Exploiting silhouettes and wearable signals for subject identification in multi-person environments. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1599–1607 (2019)
Mitra, R., Marchitto, T., Ge, Q., Zhong, B., Kanakiya, B., Cook, M., Fehrenbacher, J., Ortiz, J., Tripati, A., Lobaton, E.: Automated species-level identification of planktic foraminifera using convolutional neural networks, with comparison to human performance. Mar. Micropaleontol. 147, 16–24 (2019)
Pacifico, L.D.S., Macario, V., Oliveira, J.F.L.: Plant classification using artificial neural networks. In: IJCNN, pp. 1–6 (2018)
Ravelo, A.C., Hillaire-Marcel, C.: The use of oxygen and carbon isotopes of foraminifera in paleoceanography. In: Developments in Marine Geology, vol. 1. Elsevier (2007)
Renaud, S., Schmidt, D.: Habitat tracking as a response of the planktic foraminifer Globorotalia truncatulinoides to environmental fluctuations during the last 140 kyr. Mar. Micropaleontol. 49, 97–122 (2003)
Reynolds, D.A.: Gaussian mixture models. Encyclopedia Biometrics (2009)
Rillo, M.C., Whittaker, J., Ezard, T.H., Purvis, A., Henderson, A., Stukins, S., Miller, C.: The unknown planktonic foraminiferal pioneer Henry A. Buckley and his collection at The Natural History Museum. J. Micropalaeontol. 36, 191–194 (2016)
Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016)
Schmidt, D., Renaud, S., Bollmann, J., Schiebel, R., Thierstein, H.: Size distribution of Holocene planktic foraminifer assemblages: biogeography, ecology and adaptation. Mar. Micropaleontol. 50, 319–338 (2004)
Schneider, S., Taylor, G.W., Linquist, S.S., Kremer, S.C.: Similarity learning networks for animal individual re-identification - beyond the capabilities of a human observer. WACV abs/1902.09324 (2019). http://arxiv.org/abs/1902.09324
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a united embedding for face recognition and clustering. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2, 815–823 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)
Sluys, R.: The unappreciated, fundamentally analytical nature of taxonomy and the implications for the inventory of biodiversity. Biodivers. Conserv. 22, 1095–1105 (2013)
Szegedy, C., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
Tan, D., Ang, Y., Lim, G., Ismail, M., Meier, R.: From ‘cryptic species’ to integrative taxonomy: an iterative process involving DNA sequences, morphology, and behaviour leads to the resurrection of Sepsis pyrrhosoma (sepsidae: Diptera). Zoolog. Scr. 39, 51–61 (2010)
Tuia, D., et al.: Seeing biodiversity: perspectives in machine learning for wildlife conservation. arXiv preprint arXiv:2110.12951 (2021)
Vaswani, A., et al.: Attention is all you need. CoRR 1706, 03762 (2017)
Weller, A., Harris, A., Ware, J.: Two supervised neural networks for classification of sedimentary organic matter images from palynological preparations. Math. Geol. 39, 657–671 (2007)
Wägele, H., et al.: The taxonomist - an endangered race. A practical proposal for its survival. Front. Zool. 8, 25 (2011)
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks (2017)
Acknowledgements
TK was supported by the UKRI CDT in Interactive Artificial Intelligence under the grant EP/S022937/1. AYH was supported by VR grant 2020-03515. DNS was supported by NERC grant NE/P019439/1. We thank R Marchant and his team for making available source code and testing regime details to compare to [30]. Thanks to M Lagunes-Fortiz and W Andrew for permitting use and adaptation of source code related to metric learning.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Karaderi, T., Burghardt, T., Hsiang, A.Y., Ramaer, J., Schmidt, D.N. (2022). Visual Microfossil Identification via Deep Metric Learning. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13363. Springer, Cham. https://doi.org/10.1007/978-3-031-09037-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-09037-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09036-3
Online ISBN: 978-3-031-09037-0
eBook Packages: Computer ScienceComputer Science (R0)