Abstract
Growing demands for automated analysis of animal behavior in areas such as neuroscience, psychology, genetics and pharmacology have been witnessed in recent decades. Some progresses have been made, but studies on social behavior analysis, which is more challenging, are rarely seen and almost all of them rely on hand-crafted features. Motivated by the concept of word embedding in NLP and the success of deep learning, we present a method that extracts features for both of the mouse agents involved in social behavior events and the scenario context using embedding networks, then uses an LSTM network to model the behaviors based on the agent and context embeddings. Our method is tested on a novel dataset, RatSI [8]. We find our mouse state embedding method outperforms traditional hand-crafted feature based methods.
Student paper.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Burgos-Artizzu, X.P., Dollár, P., Lin, D., Anderson, D.J., Perona, P.: Social behavior recognition in continuous video. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1322–1329. IEEE (2012)
Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72. IEEE (2005)
File, S.E., Seth, P.: A review of 25 years of the social interaction test. Eur. J. Pharmacol. 463(1–3), 35–53 (2003)
Giancardo, L., et al.: Automatic visual tracking and social behaviour analysis with multiple mice. PLoS ONE 8(9), e74557 (2013)
Jhuang, H., et al.: Automated home-cage behavioural phenotyping of mice. Nat. Commun. 1, 68 (2010)
Ji, S., Xu, W., Yang, M., Yu, K.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)
Lewejohann, L., Hoppmann, A.M., Kegel, P., Kritzler, M., Krüger, A., Sachser, N.: Behavioral phenotyping of a murine model of alzheimer’s disease in a seminaturalistic environment using RFID tracking. Behav. Res. Methods 41(3), 850–856 (2009)
Lorbach, M., Kyriakou, E.I., Poppe, R., van Dam, E.A., Noldus, L.P., Veltkamp, R.C.: Learning to recognize rat social behavior: Novel dataset and cross-dataset application. J. Neurosci. Methods 300, 166–172 (2017)
Peters, S.M., Pothuizen, H.H., Spruijt, B.M.: Ethological concepts enhance the translational value of animal models. Eur. J. Pharmacol. 759, 42–50 (2015)
Ren, Z., Annie, A.N., Ciernia, V., Lee, Y.J.: Who moved my cheese? Automatic annotation of rodent behaviors with convolutional neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1277–1286. IEEE (2017)
Rousseau, J., Van Lochem, P., Gispen, W., Spruijt, B.: Classification of rat behavior with an image-processing method and a neural network. Behav. Res. Methods Instrum. Comput. 32(1), 63–71 (2000)
Salem, G.H., et al.: SCORHE: a novel and practical approach to video monitoring of laboratory mice housed in vivarium cage racks. Behav. Res. Methods 47(1), 235–250 (2015)
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems, pp. 568–576 (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Urbach, Y.K., Bode, F.J., Nguyen, H.P., Riess, O., von Hörsten, S.: Neurobehavioral tests in rat models of degenerative brain diseases. In: Anegon, I. (ed.) Rat Genomics. MIMB, vol. 597, pp. 333–356. Humana Press, Totowa (2010). https://doi.org/10.1007/978-1-60327-389-3_24
Wang, Z., Mirbozorgi, S.A., Ghovanloo, M.: An automated behavior analysis system for freely moving rodents using depth image. Med. Biol. Eng. Comput. 56, 1–15 (2018)
Zhang, Z., Yingchun, Y., Zhaohui, W., Qian, M.: A posture recognition system for rat cyborg automated navigation. Chin. J. Electron. 27(4), 687–693 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Z., Yang, Y., Wu, Z. (2019). Social Behavior Recognition in Mouse Video Using Agent Embedding and LSTM Modelling. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11858. Springer, Cham. https://doi.org/10.1007/978-3-030-31723-2_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-31723-2_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31722-5
Online ISBN: 978-3-030-31723-2
eBook Packages: Computer ScienceComputer Science (R0)