skip to main content
10.1145/1553374.1553483acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlConference Proceedingsconference-collections
research-article

Sparse higher order conditional random fields for improved sequence labeling

Published:14 June 2009Publication History

ABSTRACT

In real sequence labeling tasks, statistics of many higher order features are not sufficient due to the training data sparseness, very few of them are useful. We describe Sparse Higher Order Conditional Random Fields (SHO-CRFs), which are able to handle local features and sparse higher order features together using a novel tractable exact inference algorithm. Our main insight is that states and transitions with same potential functions can be grouped together, and inference is performed on the grouped states and transitions. Though the complexity is not polynomial, SHO-CRFs are still efficient in practice because of the feature sparseness. Experimental results on optical character recognition and Chinese organization name recognition show that with the same higher order feature set, SHO-CRFs significantly outperform previous approaches.

References

  1. Collins, M. (2002a). Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. Proceedings of Empirical Methods in Natural Language Processing (pp. 1--8). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Collins, M. (2002b). Ranking algorithms for named entity extraction: Boosting and the voted perceptron. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (pp. 489--496). Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Galassi, U., Giordana, A., & Saitta, L. (2007). Structured hidden markov model: A general framework for modeling complex sequences. AI*IA 2007: Artificial Intelligence and Human-Oriented Computing (pp. 290--301). Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Jin, G., & Chen, X. (2008). The fourth international chinese language processing bakeoff: Chinese word segmentation, named entity recognition and chinese pos tagging. Proceedings of Sixth Special Interest Group on Chinese Language Processing Workshop (pp. 69--81).Google ScholarGoogle Scholar
  5. Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the 18th International Conference on Machine Learning (pp. 282--289). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Roth, D., & tau Yih, W. (2005). Integer linear programming inference for conditional random fields. Proceedings of the 22nd International Conference on Machine learning. (pp. 736--743). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Sarawagi, S., & Cohen, W. (2004). Semi-markov conditional random fields for information extraction. Advances in Neural Information Processing Systems (pp. 1185--1192).Google ScholarGoogle Scholar
  8. Taskar, B., Guestrin, C., & Koller, D. (2003). Max-margin markov networks. Advances in Neural Information Processing Systems (pp. 25--32).Google ScholarGoogle Scholar
  9. Yang, F., Zhao, J., & Zou, B. (2008). CRFs-based named entity recognition incorporated with heuristic entity list searching. Proceedings of Sixth Special Interest Group on Chinese Language Processing Workshop (pp. 171--174).Google ScholarGoogle Scholar
  10. Yu, X., Lam, W., Chan, S.-K., Wu, Y., & Chen, B. (2008). Chinese NER using CRFs and logic for the fourth sighan bakeoff. Proceedings of Sixth Special Interest Group on Chinese Language Processing Workshop (pp. 102--105).Google ScholarGoogle Scholar

Index Terms

  1. Sparse higher order conditional random fields for improved sequence labeling

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in
              • Published in

                cover image ACM Other conferences
                ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning
                June 2009
                1331 pages
                ISBN:9781605585161
                DOI:10.1145/1553374

                Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 14 June 2009

                Permissions

                Request permissions about this article.

                Request Permissions

                Check for updates

                Qualifiers

                • research-article

                Acceptance Rates

                Overall Acceptance Rate140of548submissions,26%

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader