Abstract
A latent variable conditional random fields (CRF) model is proposed to improve sequence labeling, which utilizes the BIO encoding schema as latent variable to capture the latent structure of hidden variables and observation data. The proposed model automatically selects the best encoding schema for each given input sequence. Through experimentation, it is demonstrated that the proposed model unveils the latent variable while performing robustly on sequence-labeling prediction tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baum, L.E.: An inequality and associated maximization technique in statistical estimation of probabilistic functions of a markov process. Inequalities 3, 1–8 (1972)
Baum, L.E., Eagon, J.A.: An inequality with applications to statistical estimation for probabilistic functions of markov processes and to a model for ecology. Bull. Am. Math. Soc. 37(3), 360–363 (1967)
Baum, L.E., Petrie, T.: Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Stat. 37(6), 1554–1563 (1966)
Berger, A.L., Pietra, S.A.D., Pietra, V.J.D.: A maximum entropy approach to natural language processing. Comput. Linguist. 22(1), 39–71 (1996)
Cuong, N.V., Ye, N., Lee, W.S., Chieu, H.L.: Conditional random field with high-order dependencies for sequence labeling and segmentation. J. Mach. Learn. Res. 15(1), 981–1009 (2014)
Dai, H., Lai, P., Chang, Y., Tsa, R.T.: Enhancing of chemical compound and drug name recognition using representative tag scheme and fine-grained tokenization. J. Cheminformatics 7(1), 1–10 (2015)
Fine, S., Singer, Y., Tishby, N.: The hierarchical hidden Markov model: analysis and applications. Mach. Learn. 32(1), 41–62 (1998)
Guo, S., Chang, M.W., Kiciman, E.: To link or not to link? a study on end-to-end tweet entity linking. In: The Conference of the North American Chapter of the Association of Computational Linguistics, pp. 1020–1030 (2013)
Gupta , P., Andrassy, B.: Table filling multi-task recurrent neural network for joint entity and relation extraction. In: The International Conference on Computational Linguistics, pp. 2537–2547 (2016)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging (2015). http://arxiv.org/abs/1508.01991s
Lafferty, J.D., Mccallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: The Eighteenth International Conference on Machine Learning, pp. 282–289 (2001)
Liu, Y., Che, W., Guo, J., Bin, Q., Liu, T.: Exploring segment representations for neural segmentation models. In: The International Joint Conference on Artificial Intelligence, pp. 2880–288 (2016)
Lu, J., Venugopal, D., Gogate, V., Ng, V.: Joint inference for event coreference resolution. In: The International Conference on Computational Linguistics, pp. 3264–3275 (2016)
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF. In: The Annual Meeting of the Association for Computational Linguistics, pp. 1064–1074 (2016)
McCallum, A., Freitag, D., Pereira, F.C.N.: Maximum entropy Markov models for information extraction and segmentation. In: The International Conference on Machine Learning, pp. 591–598 (1999)
Mintz, M., Bills, R.S.S., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: The Annual Meeting of the Association for Computational Linguistics, pp. 1003–1011 (2009)
Muis, A.O., Lu, W.: Weak semi-Markov CRFS for noun phrase chunking in informal text. In: The North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 714–719 (2016)
Nguyen, V.C., Lee, W.S., Ye, N., Hai, L.C.: Semi-Markov conditional random field with high-order feature, pp. 1–4 (2011)
Okanohara, D., Miyao, Y., Tsuruoka, Y., Tisuji, J.: Improving the scalability of semi-Markov conditional random fields for named entity recognition. In: The Annual Meeting of the Association for Computational Linguistics, pp. 465–472 (2006)
Petrov, S., Dan, K.: Sparse multi-scale grammars for discriminative latent variable parsing. In: The Conference on Empirical Methods in Natural Language Processing, pp. 867–876 (2008)
Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: The Conference on Computational Natural Language Learning, pp. 147–155 (2009)
Ratnaparkhi, A.: A maximum entropy model for part-of-speech tagging. In: The Conference on Empirical Methods in Natural Language Processing, pp. 133–142 (1996)
Rei, M., Crichton, G.K.O., Pyysalo, S.: Attending to characters in neural sequence labeling models (2016). http://arxiv.org/abs/1611.04361
Rosenberg, D.S., Dan, K., Taskar, B.: Mixture-of-parents maximum entropy Markov models (2012). http://arxiv.org/abs/1206.5261
Sarawagi, S., Cohen, W.W.: Semi-Markov conditional random fields for information extraction. In: The Neural Information Processing Systems, pp. 1185–1192 (2004)
Sun, X., Huang, D., Ren, F.: Detecting new words from chinese text using latent semi-CRF models. IEICE Trans. Inform. Syst. 93(6), 1386–1393 (2010)
Sun, X., Nan, X.: Chinese base phrases chunking based on latent semi-CRF mode. In: The International Conference on Natural Language Processing and Knowledge Engineering, pp. 1–7 (2010)
Tseng, H., Chang, P., Andrew, G., Jurafsky, D., Manning, C.: Sequential labeling with latent variables. In: The Workshop on Chinese Language Processing, pp. 168–171 (2015)
Zhang, H.P., Liu, Q., Cheng, X.Q., Zhang, H., Yu, H.K.: Chinese lexical analysis using hierarchical hidden Markov model. In: The Workshop on Chinese Language Processing, pp. 63–70 (2003)
Zhao, H., Huang, C.N., Li, M., Kudo, T.: An improved Chinese word segmentation system with conditional random field. In: The Workshop on Chinese Language Processing, pp. 162–165 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lin, J.CW., Wu, J.MT., Shao, Y., Pirouz, M., Zhang, B. (2019). A Latent Variable CRF Model for Labeling Prediction. In: Lin, JW., Ting, IH., Tang, T., Wang, K. (eds) Multidisciplinary Social Networks Research. MISNC 2019. Communications in Computer and Information Science, vol 1131. Springer, Singapore. https://doi.org/10.1007/978-981-15-1758-7_6
Download citation
DOI: https://doi.org/10.1007/978-981-15-1758-7_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1757-0
Online ISBN: 978-981-15-1758-7
eBook Packages: Computer ScienceComputer Science (R0)