Abstract
Similarity is a fundamental concept in artificial intelligence and cognitive sciences. Despite all the efforts made to study similarity, defining and measuring it between concepts or images remains challenging. Fortunately, measuring similarity is comparable to answering “why”/“in which respects” two stimuli are similar. While most related works done in computer sciences try to measure the similarity, we propose to analyze it from a different angle and retrospectively find such respects. In this paper, we provide a pipeline allowing us to find, for a given dataset of image pairs, what are the different concepts generally compared and why. As an indirect evaluation, we propose generating an automatic explanation of clusters of image pairs found using a model pre-trained on texts/images. An experimental study highlights encouraging results toward a better comprehension of visual similarity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Rissland, E.: AI and similarity. IEEE Intell. Syst. 21, 39–49 (2006)
Tversky, A., Gati, I.: Studies of similarity. In: Cognition and Categorization (1978)
Gentner, D., Markman, A.B.: Structure mapping in analogy and similarity. Am. Psychol. 52, 45–56 (1997)
Tirilly, P., Mu, X., Huang, C., Xie, I., Jeong, W., Zhang, J.: On the consistency and features of image similarity. In: IIiX, pp. 164–173 (2012)
Liu, T., Cooper, L.A.: The influence of task requirements on priming in object decision and matching. Mem. Cognit. 29, 874–882 (2001)
Goodman, N.: Seven strictures on similarity. In: Problems and Projects (1972)
Medin, D.L., Goldstone, R.L., Gentner, D.: Respects for similarity. Psychol. Rev. 100, 254–274 (1993)
Peterson, J.C., Abbott, J.T., Griffiths, T.L.: Evaluating (and improving) the correspondence between deep neural networks and human representations. Cogn. Sci. 42, 2648–2669 (2018)
Zheng, C.Y., Pereira, F., Baker, C.I., Hebart, M.N.: Revealing interpretable object representations from human behavior. In: ICLR (2019)
Hebart, M.N., Zheng, C.Y., Pereira, F., Baker, C.I.: Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nat. Hum. Behav. 4, 1173–1185 (2020)
Kubilius, J., Bracci, S., Op de Beeck, H.P.: Deep neural networks as a model for human shape sensitivity. PLoS Comput. Biol. 12, e1004896 (2016)
Pramod, R.T., Arun, S.P.: Do computational models differ systematically from human object perception? In: CVPR, pp. 1601–1609 (2016)
Sadovnik, A., Gharbi, W., Vu, T., Gallagher, A.C.: Finding your lookalike: measuring face similarity rather than face identity. In: CVPR Workshops, pp. 2345–2353 (2018)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The effectiveness of deep features as a perceptual metric. In: CVPR, pp. 586–595 (2018)
German, J.S., Jacobs, R.A.: Can machine learning account for human visual object shape similarity judgments? Vis. Res. 167, 87–99 (2020)
Rosenfeld, A., Solbach, M.D., Tsotsos, J.K.: Totally looks like - how humans compare, compared to machines. In: ACCV, pp. 282–297 (2018)
Hamilton, M., Fu, S., Freeman, W.T., Lu, M.: Conditional image retrieval. CoRR, abs/2007.07177 (2020)
Attarian, M., Roads, B.D., Mozer, M.C.: Transforming neural network visual representations to predict human judgments of similarity. In: SVRHM Workshop (2020)
Chen, C., Li, O., Barnett, A., Su, J., Rudin, C.: This looks like that: deep learning for interpretable image recognition. CoRR, abs/1806.10574 (2018)
Nauta, M., Jutte, A., Provoost, J.C., Seifert, C.: This looks like that, because ... explaining prototypes for interpretable image recognition. CoRR, abs/2011.02863 (2020)
Risser-Maroix, O., Kurtz, C., Loménie, N.: Learning an adaptation function to assess image visual similarities. In: ICIP, pp. 2498–2502 (2021)
Takmaz, A., Probst, T., Pani Paudel, D., Van Gool, L.: Learning feature representations for look-alike images. In: CVPR Workshops (2019)
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., et al.: Learning transferable visual models from natural language supervision. In: ICML, vol. 139, pp. 8748–8763 (2021)
Cichy, R.M., Kaiser, D.: Deep neural networks as scientific models. Trends Cogn. Sci. 23, 305–317 (2019)
Zhuang, C., et al.: Unsupervised neural network models of the ventral visual stream. Proc. Natl. Acad. Sci. 118, 237–250 (2021)
Lake, B., Zaremba, W., Fergus, R., Gureckis, T.: Neural networks predict category typicality ratings for images. In: CogSci, pp. 1–15 (2015)
Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: ICCV, pp. 3018–3027 (2017)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Risser-Maroix, O., Kurtz, C., Loménie, N. (2022). Discovering Respects for Visual Similarity. In: Krzyzak, A., Suen, C.Y., Torsello, A., Nobile, N. (eds) Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2022. Lecture Notes in Computer Science, vol 13813. Springer, Cham. https://doi.org/10.1007/978-3-031-23028-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-23028-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23027-1
Online ISBN: 978-3-031-23028-8
eBook Packages: Computer ScienceComputer Science (R0)