Abstract
Deception is a common feature in our daily life, which can be recognised by thermal imaging. Previous research has attempted to identify deception with causality features extracted from thermal images using the extended Granger causality (eGC) method. As the eGC transformation is complicated, in this paper we explore whether a transfer learning model trained on the eGC-transformed thermal deception dataset can be applied to the original thermal data to recognise deception. We explore two feature selection methods, namely linear discriminant analysis (LDA) and t-distributed random neighborhood embedding (t-SNE), and three classifiers, including a support vector machine (SVM), a feed forward neural network (NN) and a bidirectional neural network (BDNN). We find that using features selected by LDA, a transfer learning NN is able to recognise deception with an accuracy of 91.7% and an F1 score of 0.92. We believe this study helps foster a deeper understanding of eGC and provides a foundation for building transfer learning models for deception recognition.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abouelenien, M., Pérez-Rosas, V., Mihalcea, R., Burzo, M.: Deception detection using a multimodal approach. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 58–65 (2014)
Anand, R., Mehrotra, K.G., Mohan, C.K., Ranka, S.: An improved algorithm for neural network classification of imbalanced training sets. IEEE Trans. Neural Netw. 4(6), 962–969 (1993)
Bressler, S.L., Seth, A.K.: Wiener-granger causality: a well established methodology. Neuroimage 58(2), 323–329 (2011)
Brzezinski, R.Y., et al.: Automated thermal imaging for the detection of fatty liver disease. Sci. Rep. 10(1), 1–11 (2020)
Derakhshan, A., Mikaeili, M., Nasrabadi, A.M., Gedeon, T.: Network physiology of ‘fight or flight’ response in facial superficial blood vessels. Physiol. Meas. 40(1), 014002 (2019)
Gold, C., Sollich, P.: Model selection for support vector machine classification. Neurocomputing 55(1–2), 221–249 (2003)
Iyanda, A.R., Ninan, O.D., Ajayi, A.O., Anyabolu, O.G.: Predicting student academic performance in computer science courses: a comparison of neural network models. Int. J. Mod. Educ. Comput. Sci. 10(6), 1–9 (2018)
Jacobsen, C., Fosgaard, T.R., Pascual-Ezama, D.: Why do we lie? A practical guide to the dishonesty literature. J. Econ. Surv. 32(2), 357–387 (2018)
Kheiralipour, K., Ahmadi, H., Rajabipour, A., Rafiee, S., Javan-Nikkhah, M., Jayas, D.: Development of a new threshold based classification model for analyzing thermal imaging data to detect fungal infection of pistachio kernel. Agric. Res. 2(2), 127–131 (2013)
Li, M.A., Luo, X.Y., Yang, J.F.: Extracting the nonlinear features of motor imagery EEG using parametric t-SNE. Neurocomputing 218, 371–381 (2016)
Lin, Y.P., Jung, T.P.: Improving EEG-based emotion classification using conditional transfer learning. Front. Hum. Neurosci. 11, 334 (2017)
Murphy, K.P., et al.: Naive Bayes classifiers. Univ. Br. Columbia 18(60), 1–8 (2006)
Nejad, A.F., Gedeon, T.D.: Bidirectional neural networks and class prototypes. In: Proceedings of ICNN’95-International Conference on Neural Networks, vol. 3, pp. 1322–1327. IEEE (1995)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
Patro, S., Sahu, K.K.: Normalization: a preprocessing stage. arXiv preprint arXiv:1503.06462 (2015)
Pavlidis, I., Eberhardt, N.L., Levine, J.A.: Seeing through the face of deception. Nature 415(6867), 35–35 (2002)
Pavlidis, I., Levine, J.: Thermal image analysis for polygraph testing. IEEE Eng. Med. Biol. Mag. 21(6), 56–64 (2002)
Rakitianskaia, A., Bekker, E., Malan, K.M., Engelbrecht, A.: Analysis of error landscapes in multi-layered neural networks for classification. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 5270–5277. IEEE (2016)
Rosenstein, M.T., Marx, Z., Kaelbling, L.P., Dietterich, T.G.: To transfer or not to transfer. In: NIPS 2005 workshop on transfer learning, vol. 898, pp. 1–4 (2005)
Schiatti, L., Nollo, G., Rossato, G., Faes, L.: Extended granger causality: a new tool to identify the structure of physiological networks. Physiol. Meas. 36(4), 827 (2015)
Song, F., Mei, D., Li, H.: Feature selection based on linear discriminant analysis. In: 2010 International Conference on Intelligent System Design and Engineering Application, vol. 1, pp. 746–749. IEEE (2010)
Warmelink, L., Vrij, A., Mann, S., Leal, S., Forrester, D., Fisher, R.P.: Thermal imaging as a lie detection tool at airports. Law Hum. Behav. 35(1), 40–48 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Qin, Z., Zhu, X., Gedeon, T. (2021). Examining Transfer Learning with Neural Network and Bidirectional Neural Network on Thermal Imaging for Deception Recognition. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-92310-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92309-9
Online ISBN: 978-3-030-92310-5
eBook Packages: Computer ScienceComputer Science (R0)