Abstract
In this work, we want to learn to model the dynamics of similar yet distinct groups of interacting objects. These groups follow some common physical laws that exhibit specificities that are captured through some vectorial description. We develop a model that allows us to do conditional generation from any such group given its vectorial description. Unlike previous work on learning dynamical systems that can only do trajectory completion and require a part of the trajectory dynamics to be provided as input in generation time, we do generation using only the conditioning vector with no access to generation time’s trajectories. We evaluate our model in the setting of modeling human gait and, in particular pathological human gait.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The first part of that trajectory will always be real data, even at test time, directly coming from the input trajectory as we will soon explain.
References
Aliakbarian, M.S., Saleh, F., Salzmann, M., Petersson, L., Gould, S.: A stochastic conditioning scheme for diverse human motion prediction. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5222–5231 (2020)
Battaglia, P., Pascanu, R., Lai, M., Rezende, D., Kavukcuoglu, K.: Interaction networks for learning about objects, relations and physics. In: Advances in Neural Information Processing Systems, pp. 4509–4517. Neural information processing systems foundation (2016)
Battaglia, P.W., et al.: Relational inductive biases, deep learning, and graph networks. arXiv (2018)
Chang, M.B., Ullman, T., Torralba, A., Tenenbaum, J.B.: A compositional object-based approach to learning physical dynamics. In: International Conference on Learning Representations (ICLR) (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP 2014–2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp. 1724–1734. Association for Computational Linguistics (ACL) (2014). https://doi.org/10.3115/v1/d14-1179
Fragkiadaki, K., Levine, S., Felsen, P., Malik, J.: Recurrent network models for human dynamics. In: IEEE International Conference on Computer Vision (ICCV) (2015)
Gopalakrishnan, A., Mali, A., Kifer, D., Giles, C.L., Ororbia, A.G.: A neural temporal model for human motion prediction. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2019, pp. 12108–12117 (2018)
Guttenberg, N., Virgo, N., Witkowski, O., Aoki, H., Kanai, R.: Permutation-equivariant neural networks applied to dynamics prediction. arXiv (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Hoshen, Y.: VAIN: attentional multi-agent predictive modeling. In: Advances in Neural Information Processing Systems, December 2017, pp. 2702–2712. Neural information processing systems foundation (2017)
Jain, A., Zamir, A.R., Savarese, S., Saxena, A.: Structural-RNN: deep learning on spatio-temporal graphs. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, December 2016, pp. 5308–5317. IEEE Computer Society (2016). https://doi.org/10.1109/CVPR.2016.573
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR (2015)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: International Conference on Learning Representations (ICLR) (2014)
Kipf, T., Fetaya, E., Wang, K.C., Welling, M., Zemel, R.: Neural relational inference for interacting systems. In: International Conference on Machine Learning (ICML), pp. 2688–2697 (2018)
Lehrmann, A.M., Gehler, P.V., Nowozin, S.: Efficient nonlinear Markov models for human motion. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1314–1321. IEEE Computer Society (2014). https://doi.org/10.1109/CVPR.2014.171
Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., Tian, Q.: Symbiotic graph neural networks for 3D skeleton-based human action recognition and motion prediction. arXiv (2019)
Liu, Z., et al.: Towards natural and accurate future motion prediction of humans and animals. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10004–10012 (2019)
Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. In: International Conference on Learning Representations (ICLR) (2016)
Martinez, J., Black, M.J., Romero, J.: On human motion prediction using recurrent neural networks. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, January 2017, pp. 4674–4683 (2017)
Pavlovic, V., Rehg, J.M., Maccormick, J.: Learning switching linear models of human motion. In: Neural Information Processing Systems (NeurIPS), pp. 981–987 (2001)
Pitto, L., et al.: SimCP: a simulation platform to predict gait performance following orthopedic intervention in children with cerebral palsy. Front. Neurorobot. 13 (2019). https://doi.org/10.3389/fnbot.2019.00054
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986). https://doi.org/10.1038/323533a0
Santoro, A., et al.: A simple neural network module for relational reasoning. In: Advances in Neural Information Processing Systems, December 2017, pp. 4968–4977 (2017)
Shi, L., Zhang, Y., Cheng, J., Lu, H.: Skeleton-based action recognition with directed graph neural networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
van Steenkiste, S., Chang, M., Greff, K., Schmidhuber, J.: Relational neural expectation maximization: unsupervised discovery of objects and their interactions. In: 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings (2018)
Sukhbaatar, S., Szlam, A., Fergus, R.: Learning multiagent communication with backpropagation. In: Neural Information Processing Systems (NeurIPS), pp. 2244–2252 (2016)
Urtasun, R., Fleet, D.J., Geiger, A., Popović, J., Darrell, T.J., Lawrence, N.D.: Topologically-constrained latent variable models. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1080–1087. ACM Press, New York (2008). https://doi.org/10.1145/1390156.1390292
Walker, J., Marino, K., Gupta, A., Hebert, M.: The pose knows: video forecasting by generating pose futures. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 3352–3361 (2017)
Wang, B., Adeli, E., Chiu, H.K., Huang, D.A., Niebles, J.C.: Imitation learning for human pose prediction. In: Proceedings of the IEEE International Conference on Computer Vision, October 2019, pp. 7123–7132. Institute of Electrical and Electronics Engineers Inc. (2019). https://doi.org/10.1109/ICCV.2019.00722
Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Trans. Pattern Anal. Mach. Intell. 30, 283–298 (2008)
Watters, N., Tacchetti, A., Weber, T., Pascanu, R., Battaglia, P., Zoran, D.: Visual interaction networks: learning a physics simulator from video. In: Neural Information Processing Systems (NeurIPS), pp. 4539–4547 (2017)
Webb, E., Day, B., Andres-Terre, H., Lió, P.: Factorised neural relational inference for multi-interaction systems. In: ICML Workshop on Learning and Reasoning with Graph-Structured Representations (2019)
Acknowledgments
This work was supported by the Swiss National Science Foundation grant number CSSII5_177179 “Modeling pathological gait resulting from motor impairment”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Candido Ramos, J.A., Blondé, L., Armand, S., Kalousis, A. (2021). Conditional Neural Relational Inference for Interacting Systems. In: Dong, Y., Kourtellis, N., Hammer, B., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. ECML PKDD 2021. Lecture Notes in Computer Science(), vol 12979. Springer, Cham. https://doi.org/10.1007/978-3-030-86517-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-86517-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86516-0
Online ISBN: 978-3-030-86517-7
eBook Packages: Computer ScienceComputer Science (R0)
