Abstract
The Rey-Osterrieth Complex Figure Test (ROCFT) is a widely used neuropsychological tool for assessing the presence and severity of different diseases. It involves presenting a complex illustration to the patient who is asked to copy it, followed by recall from memory after 3 and 30 min. In clinical practice, a human rater evaluates each component of the reproduction, with the overall score indicating illness severity. However, this method is both time-consuming and error-prone. Efforts have been made to automate the process, but current algorithms require large-scale private datasets of up to 20,000 illustrations. With limited data, training a deep learning model is challenging. This study addresses this challenge by developing a fine-tuning strategy with multiple stages. We show that pre-training on a large-scale sketch dataset with initialized weights from ImageNet significantly reduces the mean absolute error (MAE) compared to just training with initialized weights from ImageNet, e.g., ReXNet-200 from 3.1 to 2.2 MAE. Additionally, techniques such as stochastic weight averaging (SWA) and ensembling of different architectures can further reduce the error to an MAE of 1.97.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arango-Lasprilla, J.C., et al.: Rey-osterrieth complex figure - copy and immediate recall (3 minutes): normative data for Spanish-speaking pediatric populations. NeuroRehabilitation 41(3), 593–603 (2017)
Canham, R.O., Smith, S.L., Tyrrell, A.M.: Automated scoring of a neuropsychological test: the Rey Osterrieth complex figure. In: Proceedings of the 26th Euromicro Conference. EUROMICRO 2000. Informatics: Inventing the Future, pp. 406–413. IEEE Comput. Soc (2000)
Canham, R.O., Smith, S.L., Tyrrell, A.M.: Location of structural sections from within a highly distorted complex line drawing. IEE Proc. Vis. Image Sig. Process. 152(6), 741 (2005)
Conson, M., Siciliano, M., Baiano, C., Zappullo, I., Senese, V.P., Santangelo, G.: Normative data of the Rey-Osterrieth complex figure for Italian-speaking elementary school children. Neurol. Sci. 40(10), 2045–2050 (2019)
Di Febbo, D., et al.: A decision support system for Rey-Osterrieth complex figure evaluation. Expert Syst. Appl. 213, 119226 (2023)
Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. (Proc. SIGGRAPH) 31(4), 44:1–44:10 (2012)
Han, D., Yun, S., Heo, B., Yoo, Y.: Rethinking channel dimensions for efficient model design. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 732–741 (2021)
Han, K., Wang, Y., Zhang, Q., Zhang, W., XU, C., Zhang, T.: Model Rubik’s cube: twisting resolution, depth and width for tinynets. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 19353–19364. Curran Associates, Inc. (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (June 2016)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Staff, I. (ed.) 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269. IEEE, Piscataway (July (2017)
Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., Wilson, A.G.: Averaging weights leads to wider optima and better generalization. In: Silva, R., Globerson, A., Globerson, A. (eds.) 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018, pp. 876–885. Association For Uncertainty in Artificial Intelligence (AUAI) (2018)
Jung, A.B., et al.: imgaug (2020)
Langer, N., et al.: The AI neuropsychologist: Automatic scoring of memory deficits with deep learning (2022). https://doi.org/10.1101/2022.06.15.496291
Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11976–11986 (2022)
Park, J.Y., Seo, E.H., Yoon, H.J., Won, S., Lee, K.H.: Automating Rey complex figure test scoring using a deep learning-based approach: A potential large-scale screening tool for congnitive decline (2022). https://www.researchsquare.com/article/rs-1973305/v1
Izmailov, P., Wilson, A.G.: Stochastic weight averaging in pytorch (29042019). https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/
Ross Wightman: Pytorch image models (2019)
Shin, M.S., Park, S.Y., Park, S.R., Seol, S.H., Kwon, J.S.: Clinical and empirical applications of the Rey-Osterrieth complex figure test. Nat. Protoc. 1(2), 892–899 (2006)
Simfukwe, C., An, S.S., Youn, Y.C.: Comparison of RCF scoring system to clinical decision for the Rey complex figure using machine-learning algorithm. Dement. Neurocognitive Disord. 20(4), 70–79 (2021)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR). San Diego (may 2015)
Webb, S.S., et al.: Validation of an automated scoring program for a digital complex figure copy task within healthy aging and stroke. Neuropsychology 35(8), 847–862 (2021)
Li, Y.: Development of a Haptic-based Rey-Osterrieth Complex Figure Testing and Training System with Computer Scoring and Force-feedback Rehabilitation \(\ldots \) (2010). https://repository.lib.ncsu.edu/bitstream/handle/1840.16/6062/etd.pdf?sequence=1
Zhang, X., Lv, L., Min, G., Wang, Q., Zhao, Y., Li, Y.: Overview of the complex figure test and its clinical application in neuropsychiatric disorders, including copying and recall. Front. Neurol. 12, 680474 (2021)
Acknowledgement
We thank the University Hospital Cologne for providing the data. The authors would also like to thank NVIDIA for their hardware donation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
A Augmentation Details
B Ensembling Combinations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schuster, B. et al. (2023). Multi-stage Fine-Tuning Deep Learning Models Improves Automatic Assessment of the Rey-Osterrieth Complex Figure Test. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14187. Springer, Cham. https://doi.org/10.1007/978-3-031-41676-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-41676-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-41675-0
Online ISBN: 978-3-031-41676-7
eBook Packages: Computer ScienceComputer Science (R0)
