Abstract
Diagrammatic Reasoning (DR) questions are very common in competitive examinations. However, construction of interesting and fresh DR questions can be a tedious job even for the experts. We explore the possibility of using Artificial Intelligence (AI) and computer vision (CV) for construction and solving DR problems. In this paper, we have proposed a new deep learning-based framework that can be used to solve certain types of DR problems. The research also shows that a similar framework can be used to generate new DR problems of similar characteristics. We formulate the DR problem with an extension of conventional 4\(\,\times \,\)1 Raven’s Progressive Matrix (RPM) by keeping 4 outputs. Thus, each problem sample has eight images, where the first four images are part of the input in a sequence and the last four images are options for the correct output. The first four images create a valid sequence and the target is to choose the fifth image from the next four images. To find the correct option, we have proposed a deep learning framework that consists of an LSTM, an Encoder and a fully connected classifier unit. The framework has also been used to generate new DR problems. We have tested our framework on Rotational DR problems. A new DR dataset has been generated using automated scripts to train the framework. The framework performs better as compared to SOTA deep learning frameworks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dogra, D.P., Sekh, A.A., Kar, S., Roy, P.P., Prasad, D.K.: Can we automate diagrammatic reasoning, October (2020)
Burke, H.R.: Raven’s progressive matrices: a review and critical evaluation. J. Genet. Psychol. 93(2), 199–228 (1958)
Diamantini, C., Freddi, A., Longhi, S., Potena, D., Storti, E.: A goal-oriented, ontology-based methodology to support the design of AAL environments. Expert Syst. Appl. 64, 117–131 (2016)
Zhou, Y., Sun, Y., Honavar, V.: Improving image captioning by leveraging knowledge graphs. In: IEEE Winter Conference on Applications of Computer Vision, pp. 283–293 (2019)
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C.L., Girshick, R.: CLEVR: a diagnostic dataset for compositional language and elementary visual reasoning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1988–1997 (2017)
Giorgini, P., Mylopoulos, J., Nicchiarelli, E., Sebastiani, R.: Reasoning with goal models. In: Spaccapietra, S., March, S.T., Kambayashi, Y. (eds.) ER 2002. LNCS, vol. 2503, pp. 167–181. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45816-6_22
Mineau, G.W., Godin, R.: Automatic structuring of knowledge bases by conceptual clustering. IEEE Trans. Knowl. Data Eng. 7(5), 824–829 (1995)
Raedt, L.D., Kersting, K., Natarajan, S., Poole, D.: Statistical relational artificial intelligence: logic, probability, and computation. Synth. Lect. Artif. Intell. Mach. Learn. 10(2), 1–189 (2016)
Shin, C.-U., Cha, J.-W.: End-to-end task dependent recurrent entity network for goal-oriented dialog learning. Comput. Speech Lang. 53, 12–24 (2019)
Serafini, L., d’Avila Garcez, A.S.: Learning and reasoning with logic tensor networks. In: Adorni, G., Cagnoni, S., Gori, M., Maratea, M. (eds.) AI*IA 2016. LNCS (LNAI), vol. 10037, pp. 334–348. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49130-1_25
Kazemi, S.M., Poole, D.: ReINN: a deep neural model for relational learning. In: 32nd AAAI Conference on Artificial Intelligence, pp. 6367–6375 (2018)
Garcez, A., Gori, M., Lamb, L., Serafini, L., Spranger, M., Tran, S.: Neural-symbolic computing: an effective methodology for principled integration of machine learning and reasoning. J. Appl. Logics 6(4), 611–632 (2019)
Mao, J., Gan, C., Kohli, P., Tenenbaum, J.B., Wu, J.: The neuro-symbolic concept learner: interpreting scenes, words, and sentences from natural supervision. In: International Conference on Learning Representations, pp. 1–28 (2019)
Wang, J., Wang, W., Wang, L., Wang, Z., Feng, D.D., Tan, T.: Learning visual relationship and context-aware attention for image captioning. Pattern Recogn. 98, 107075–107086 (2020)
Wang, W., Huang, Y., Wang, L.: Long video question answering: a matching-guided attention model. Pattern Recogn. 102, 107–248 (2020)
Santoro, A., Hill, F., Barrett, D., Morcos, A., Lillicrap, T.: Measuring abstract reasoning in neural networks. In: International Conference on Machine Learning, pp. 4477–4486 (2018)
Hill, F., Santoro, A., Barrett, D., Morcos, A., Lillicrap, T.: Learning to make analogies by contrasting abstract relational structure. In: International Conference on Learning Representations, pp. 1–14 (2019)
Kunda, M., McGreggor, K., Goel, A.: Addressing the ravens progressive matrices test of general intelligence. In: AAAI Fall Symposium Series, pp. 22–27 (2009)
Lovett, A., Forbus, K., Usher, J.: A structure-mapping model of raven’s progressive matrices. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 32, pp. 2761–2766 (2010)
Ragni, M., Neubert, S.: Solving Raven’s IQ-tests: an AI and cognitive modeling approach. In: Proceedings of the 20th European Conference on Artificial Intelligence, pp. 666–671. IOS Press (2012)
Lovett, A., Forbus, K.: Modeling visual problem solving as analogical reasoning., Psychol Rev. 124(1), 60 (2017)
Zhang, C., Gao, F., Jia, B., Zhu, Y., Zhu, S.-C.: Raven: a dataset for relational and analogical visual reasoning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5317–5327 (2019)
Sutskever, I., Vinyals, O., Le, Q.V.: Raven, sequence to sequence learning with neural networks. Accessed 10 Sep 2014
Bank, D., Koenigstein, N., Giryes, R.: Autoencoders, v1. Accessed 12 Mar 2020
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Choudhary, H., Dogra, D.P., Sekh, A.A. (2023). Solving Diagrammatic Reasoning Problems Using Deep Learning. In: Gupta, D., Bhurchandi, K., Murala, S., Raman, B., Kumar, S. (eds) Computer Vision and Image Processing. CVIP 2022. Communications in Computer and Information Science, vol 1777. Springer, Cham. https://doi.org/10.1007/978-3-031-31417-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-31417-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31416-2
Online ISBN: 978-3-031-31417-9
eBook Packages: Computer ScienceComputer Science (R0)