Abstract
Switching Linear Dynamical Systems (SLDS) are probabilistic graphical models used both for self-supervised segmentation and dimensionality reduction. Despite their modeling capabilities, SLDS are particularly hard to train. They oftentimes over-segment the timeseries or completely ignore some of the states, reducing the usefulness of the acquired segmentation.
To improve the segmentation in Switching Linear Dynamical Systems, we introduce explicit-duration variables with infinite support. We extend the Beam Sampling algorithm to perform the efficient inference allowing for a duration distribution with infinite support. We conduct experiments on three benchmarks (two already prevalent in the state-space model literature and one demonstrating behavior in a sparse setting) that test the correctness and efficiency of our solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We used implementation of RSLDS and SLDS provided by Linderman et al. https://github.com/lindermanlab/ssm.
References
Ansari, A.F., et al.: Deep explicit duration switching models for time series. In: NeurIPS, pp. 29949–29961 (2021)
Chiappa, S.: Explicit-Duration Markov switching models. Found. Trends ® Mach. Learn. 7(6), 803–886 (2014)
Chiappa, S., Peters, J.: Movement extraction by detecting dynamics switches and repetitions. In: NeurIPS, vol. 23. Curran Associates, Inc. (2010)
Dewar, M., Wiggins, C., Wood, F.: Inference in hidden Markov models with explicit state duration distributions. IEEE Signal Process. Lett. 19(4), 235–238 (2012)
Dong, Z., Seybold, B., Murphy, K., Bui, H.: Collapsed amortized variational inference for switching nonlinear dynamical systems. In: ICML, pp. 2638–2647 (2020)
Fox, E., Sudderth, E.B., Jordan, M.I., Willsky, A.S.: Bayesian nonparametric inference of switching dynamic linear models. IEEE Trans. Signal Process. 59(4), 1569–1585 (2011)
Johnson, M.J., Duvenaud, D.K., Wiltschko, A., Adams, R.P., Datta, S.R.: Composing graphical models with neural networks for structured representations and fast inference. In: NIPS (2016)
Johnson, M.J., Willsky, A.S.: Bayesian nonparametric hidden semi-Markov models. J. Mach. Learn. Res. 14(1), 673–701 (2013)
Kim, C.J.: Dynamic linear models with Markov-switching. J. Econ. 60(1), 1–22 (1994)
Linderman, S., Johnson, M., Miller, A., Adams, R., Blei, D., Paninski, L.: Bayesian learning and inference in recurrent switching linear dynamical systems. In: AISTATS, pp. 914–922 (2017)
Murphy, K.: Switching Kalman Filters (1998)
Nassar, J., Linderman, S.W., Bugallo, M.F., Park, I.M.: Tree-structured recurrent switching linear dynamical systems for multi-scale modeling. In: ICLR (2019)
Neal, R.M.: Slice sampling. Ann. Stat. 31(3), 705–767 (2003)
Oh, S., Rehg, J., Balch, T., Dellaert, F.: Learning and inferring motion patterns using parametric segmental switching linear dynamic systems. Int. J. Comput. Vis. 77 (2008)
Van Gael, J., Saatci, Y., Teh, Y.W., Ghahramani, Z.: Beam sampling for the infinite hidden Markov model. In: ICML, pp. 1088–1095 (2008)
Wang, Y., Blei, D., Cunningham, J.P.: Posterior collapse and latent variable non-identifiability. In: NeurIPS, pp. 5443–5455 (2021)
Yu, S.Z.: Hidden Semi-Markov Models: Theory, Algorithms and Applications. Computer Science Reviews and Trends. Elsevier, Amsterdam; Boston (2016)
Zoltowski, D., Pillow, J., Linderman, S.: A general recurrent state space framework for modeling neural dynamics during decision-making. In: ICML, pp. 11680–11691 (2020)
Acknowledgement
This work was supported by the Polish National Science Centre (NCN) under grant OPUS-18 no. 2019/35/B/ST6/04379.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Słupiński, M., Lipiński, P. (2024). Improving SLDS Performance Using Explicit Duration Variables with Infinite Support. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1963. Springer, Singapore. https://doi.org/10.1007/978-981-99-8138-0_10
Download citation
DOI: https://doi.org/10.1007/978-981-99-8138-0_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8137-3
Online ISBN: 978-981-99-8138-0
eBook Packages: Computer ScienceComputer Science (R0)