Abstract
The pareidolia phenomenon is a discriminating characteristic of psychiatric disorders, expressed through visual illusions seen by patients. Typically, it can be diagnosed through the noise pareidolia test, which is time-consuming to both patients and experts. In this research, we propose a novel computer-assisted method to identify pareidolia phenomenon. The idea is to emulate patient behavior in face detection models to get a similar behavior in noise pareidolia tests as patients. Unlike most medical image analysis methods, for psychiatric disorders the ground-truth varies from patient to patient, making this challenging. For a set of training patients, we fine-tune reference models to detect noise pareidolia test responses in the same way as each individual patient. Then, a new test patient is identified by comparing their behavior to the reference models using a distance function in a trained embedding space. In the experiments, the effectiveness of the proposed method is demonstrated. Further, we can show that our method can improve the efficiency of the clinical noise pareidolia test by reducing the number of necessary test images while reaching a comparable high accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abbas, A., Chalup, S.: From face recognition to facial pareidolia: analysing hidden neuron activations in CNNs for cross-depiction recognition. In: 2019 International Joint Conference on Neural Networks, pp. 1–8 (2019)
Ballard, C., et al.: A detailed phenomenological comparison of complex visual hallucinations in dementia with Lewy bodies and Alzheimer’s disease. Int. Psychogeriatr. 9(4), 381–388 (1997)
Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 539–546 (2005)
Dwyer, D.B., Falkai, P., Koutsouleris, N.: Machine learning approaches for clinical psychology and psychiatry. Annu. Rev. Clin. Psychol. 14, 91–118 (2018)
Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1735–1742 (2006)
Ker, J., Wang, L., Rao, J., Lim, T.: Deep learning applications in medical image analysis. IEEE Access 6, 9375–9389 (2017)
Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018)
Klöppel, S., et al.: Accuracy of dementia diagnosis-a direct comparison between radiologists and a computerized method. Brain 131(11), 2969–2974 (2008)
Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Mamiya, Y., et al.: The pareidolia test: a simple neuropsychological test measuring visual hallucination-like illusions. PLoS ONE 11(5), e0154713 (2016)
Natsume, R., Inoue, K., Fukuhara, Y., Yamamoto, S., Morishima, S., Kataoka, H.: Understanding fake faces. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11131, pp. 566–576. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11015-4_42
Perlin, K.: An image synthesizer. ACM SIGGRAPH Comput. Graph. 19(3), 287–296 (1985)
Pettersson-Yeo, W., et al.: Using genetic, cognitive and multi-modal neuroimaging data to identify ultra-high-risk and first-episode psychosis at the individual level. Psychol. Med. 43(12), 2547–2562 (2013)
Rajkomar, A., Dean, J., Kohane, I.: Machine learning in medicine. N. Engl. J. Med. 380(14), 1347–1358 (2019)
Sajda, P.: Machine learning for detection and diagnosis of disease. Annu. Rev. Biomed. Eng. 8, 537–565 (2006)
Uchiyama, M., et al.: Pareidolias: complex visual illusions in dementia with Lewy bodies. Brain 135(8), 2458–2469 (2012)
Xu, Y., et al.: Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinform. 18(1), 1–17 (2017)
Yang, S., Luo, P., Loy, C.C., Tang, X.: Wider face: a face detection benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5525–5533 (2016)
Yokoi, K., Nishio, Y., Uchiyama, M., Shimomura, T., Iizuka, O., Mori, E.: Hallucinators find meaning in noises: pareidolic illusions in dementia with Lewy bodies. Neuropsychologia 56, 245–254 (2014)
Zhou, L.F., Meng, M.: Do you see the “face’’? Individual differences in face pareidolia. J. Pac. Rim. Psychol. 14, e2 (2020)
Acknowledgments
We would like to thank researchers at the Integrated Innovation Lab for Psychiatry, Keio University School of Medicine for providing us expert knowledge and dataset annotations for pareidolia phenomenon. This work was supported by JST, CREST Grant Number JPMJCR1686, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, Z., Kastner, M.A., Satoh, S. (2022). On Assisting Diagnoses of Pareidolia by Emulating Patient Behavior. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13141. Springer, Cham. https://doi.org/10.1007/978-3-030-98358-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-98358-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98357-4
Online ISBN: 978-3-030-98358-1
eBook Packages: Computer ScienceComputer Science (R0)