Abstract
Traditional Hidden Markov Models (HMM) allow us to discover the latent structure of the observed data (both discrete and continuous). Recently proposed DenseHMM provides hidden states embedding and uses the co-occurrence-based learning schema. However, it is limited to discrete emissions, which does not meet many real-world problems. We address this shortcoming by discretizing observations and using a region-based co-occurrence matrix in the training procedure. It allows embedding hidden states for continuous emission problems and reducing the training time for large sequences. An application of the proposed approach concerns recommender systems, where we try to explain how the current interest of a given user in a given group of products (current state of the user) influences the saturation of the list of recommended products with the group of products. Computational experiments confirmed that the proposed approach outperformed regular HMMs in several benchmark problems. Although the emissions are estimated roughly, we can accurately infer the states.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
https://hmmlearn.readthedocs.io/en/latest/, Last accessed 31 Mar 2023.
- 2.
https://sifter.org/simon/journal/20061211.html. Last accessed 2 Dec 2022.
References
Boeker, M., Hammer, H.L., Riegler, M.A., Halvorsen, P., Jakobsen, P.: Prediction of schizophrenia from activity data using hidden Markov model parameters. Neural Comput. Appl. 35, 5619–5630 (2022)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B (Methodol.) 39, 1–38 (1977)
Harper, F.M., Konstan, J.A.: The MovieLens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5, 19:1–19:19 (2016)
Hsiao, R., Tam, Y., Schultz, T.: Generalized Baum-Welch algorithm for discriminative training on large vocabulary continuous speech recognition system. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 3769–3772 (2009)
Huang, K., Fu, X., Sidiropoulos, N.D.: Learning hidden Markov models from pairwise co-occurrences with application to topic modeling. In: International Conference on Machine Learning, vol. 80, pp. 2073–2082 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference for Learning Representations (2015)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Lorek, P., Nowak, R., Trzcinski, T., Zieba, M.: FlowHMM: flow-based continuous hidden Markov models. In: Advances in Neural Information Processing Systems, vol. 35, pp. 8773–8784 (2022)
Sicking, J., Pintz, M., Akila, M., Wirtz, T.: DenseHMM: learning hidden markov models by learning dense representations. In: Advances in Neural Information Processing Systems (2020)
Zhang, F., Han, S., Gao, H., Wang, T.: A Gaussian mixture based hidden Markov model for motion recognition with 3D vision device. Comput. Electr. Eng. 83, 106603 (2020)
Zhang, M., Jiang, X., Fang, Z., Zeng, Y., Xu, K.: High-order hidden Markov model for trend prediction in financial time series. Phys. A 517, 1–12 (2019)
Acknowledgement
This work was supported by the Polish National Science Centre (NCN) under grant OPUS-18 no. 2019/35/B/ST6/04379.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Balcer, K., Lipinski, P. (2024). Extending DenseHMM with Continuous Emission. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14452. Springer, Singapore. https://doi.org/10.1007/978-981-99-8076-5_17
Download citation
DOI: https://doi.org/10.1007/978-981-99-8076-5_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8075-8
Online ISBN: 978-981-99-8076-5
eBook Packages: Computer ScienceComputer Science (R0)