Article

Kernel conditional random fields: representation and clique selection

Authors:

Yan LiuAuthors Info & Claims

ICML '04: Proceedings of the twenty-first international conference on Machine learning

Page 64

https://doi.org/10.1145/1015330.1015337

Published: 04 July 2004 Publication History

Abstract

Kernel conditional random fields (KCRFs) are introduced as a framework for discriminative modeling of graph-structured data. A representer theorem for conditional graphical models is given which shows how kernel conditional random fields arise from risk minimization procedures defined using Mercer kernels on labeled graphs. A procedure for greedily selecting cliques in the dual representation is then proposed, which allows sparse representations. By incorporating kernels and implicit feature spaces into conditional graphical models, the framework enables semi-supervised learning algorithms for structured data through the use of graph kernels. The framework and clique selection methods are demonstrated in synthetic data experiments, and are also applied to the problem of protein secondary structure prediction.

References

[1]

Altun, Y., Tsochantaridis, I., & Hofmann, T. (2003). Hidden Markov support vector machines. ICML'03.

[2]

Belkin, M., & Niyogi, P. (2002). Semi-supervised learning on manifolds (Technical Report TR-2002-12). University of Chicago.

[3]

Chapelle, O., Weston, J., & Schoelkopf, B. (2002). Cluster kernels for semi-supervised learning. NIPS'02.

[4]

Collins, M. (2002). Discriminative training methods for hidden Markov models: Theory and experiments with perception algorithms. Proceedings of EMNLP'02.

Digital Library

[5]

Cuff, J., & Barton, G. (1999). Evaluation and improvement of multiple sequence methods for protein secondary structure prediction. Proteins, 34, 508--519.

[6]

Della Pietra, S., Della Pietra, V., & Lafferty, J. (1997). Inducing features of random fields. IEEE PAMI, 19, 380--393.

Digital Library

[7]

Joachims, T. (1998). Text Categorization with Support Vector Machines: Learning with Many Relevant Features. ECML.

Digital Library

[8]

Joachims, T. (1999). Transductive inference for text classification using support vector machines. ICML'99.

Digital Library

[9]

Jones, D. (1999). Protein secondary structure prediction based on position-specific scoring matrices. J Mol Biol., 292, 195--202.

[10]

Kabsch, W., & Sander, C. (1983). Dictionary of protein secondary structure: Pattern recognition of hydrogen-bonded and geometrical features. Biopolymers, 22, 2577--2637.

[11]

Kim, H., & Park, H. (2003). Protein secondary structure prediction based on an improved support vector machines approach. Protein Eng., 16, 553--60.

[12]

Kimeldorf, G., & Wahba, G. (1971). Some results on Tchebychean spline functions. J. Math. Anal. Applic., 33, 82--95.

[13]

Kumar, S., & Hebert, M. (2003). Discriminative fields for modeling spatial dependencies in natural images. NIPS'03.

[14]

Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML'01.

Digital Library

[15]

Lanckriet, G., Cristianini, N., an Laurent El Ghaoui, P. B., & Jordan, M. (2004). Learning the kernel matrix with semi-definite programming. Journal of Machine Learning Research, 5, 27--72.

Digital Library

[16]

McCallum, A. (2003). Efficiently inducing features of conditional random fields. UAI'03.

Digital Library

[17]

Pinto, D., McCallum, A., Wei, X., & Croft, W. B. (2003). Table extraction using conditional random fields. SIGIR'03.

Digital Library

[18]

Sha, F., & Pereira, F. (2003). Shallow parsing with conditional random fields. Proceedings of HLT-NAACL.

Digital Library

[19]

Smola, A., & Kondor, R. (2003). Kernels and regularization on graphs. COLT'03.

[20]

Taskar, B., Guestrin, C., & Koller, D. (2003). Max-margin Markov networks. NIPS'03.

[21]

Zhu, J., & Hastie, T. (2001). Kernel logistic regression and the import vector machine. NIPS'01.

[22]

Zhu, X., Gharahmani, Z., & Lafferty, J. (2003). Semi-supervised learning using Gaussian fields and harmonic functions. ICML'03.

Cited By

Pillonetto GChen TChiuso ADe Nicolao GLjung LPillonetto GChen TChiuso ADe Nicolao GLjung L(2022)Regularization in Reproducing Kernel Hilbert SpacesRegularized System Identification10.1007/978-3-030-95860-2_6(181-246)Online publication date: 14-May-2022
https://doi.org/10.1007/978-3-030-95860-2_6
Zhang CSun STian YWang Z(2019)Structured Output Prediction Using Privileged InformationIEEE Access10.1109/ACCESS.2019.29276937(106065-106074)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2927693
Liu FLin GQiao RShen C(2018)Structured Learning of Tree Potentials in CRF for Image SegmentationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2017.269045329:6(2631-2637)Online publication date: Jun-2018
https://doi.org/10.1109/TNNLS.2017.2690453
Show More Cited By

Kernel conditional random fields: representation and clique selection
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Conditional random fields for activity recognition
AAMAS '07: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems

Activity recognition is a key component for creating intelligent, multi-agent systems. Intrinsically, activity recognition is a temporal classification problem. In this paper, we compare two models for temporal classification: hidden Markov models (HMMs)...
Hierarchical hidden conditional random fields for information extraction
LION'05: Proceedings of the 5th international conference on Learning and Intelligent Optimization

Hidden Markov Models (HMMs) are very popular generative models for time series data. Recent work, however, has shown that for many tasks Conditional Random Fields (CRFs), a type of discriminative model, perform better than HMMs. Information extraction ...
Triangular-Chain Conditional Random Fields

Sequential modeling is a fundamental task in scientific fields, especially in speech and natural language processing, where many problems of sequential data can be cast as a sequential labeling or a sequence classification. In many applications, the two ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '04: Proceedings of the twenty-first international conference on Machine learning

July 2004

934 pages

ISBN:1581138385

DOI:10.1145/1015330

Conference Chair:
Carla Brodley
Purdue University/Tufts University

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 July 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

70
Total Citations
View Citations
1,778
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pillonetto GChen TChiuso ADe Nicolao GLjung LPillonetto GChen TChiuso ADe Nicolao GLjung L(2022)Regularization in Reproducing Kernel Hilbert SpacesRegularized System Identification10.1007/978-3-030-95860-2_6(181-246)Online publication date: 14-May-2022
https://doi.org/10.1007/978-3-030-95860-2_6
Zhang CSun STian YWang Z(2019)Structured Output Prediction Using Privileged InformationIEEE Access10.1109/ACCESS.2019.29276937(106065-106074)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2927693
Liu FLin GQiao RShen C(2018)Structured Learning of Tree Potentials in CRF for Image SegmentationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2017.269045329:6(2631-2637)Online publication date: Jun-2018
https://doi.org/10.1109/TNNLS.2017.2690453
Xiao QSong R(2018)Action recognition based on hierarchical dynamic Bayesian networkMultimedia Tools and Applications10.1007/s11042-017-4614-077:6(6955-6968)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1007/s11042-017-4614-0
Sun SZhang CTian Y(2018)A New Method for Structured Learning with Privileged InformationComputational Science – ICCS 201810.1007/978-3-319-93701-4_35(453-461)Online publication date: 12-Jun-2018
https://doi.org/10.1007/978-3-319-93701-4_35
Xiao QZhao Y(2017)Action Recognition Based on Graph Model Inference2017 10th International Symposium on Computational Intelligence and Design (ISCID)10.1109/ISCID.2017.48(93-96)Online publication date: Dec-2017
https://doi.org/10.1109/ISCID.2017.48
Liu CLiu JHe ZZhai YHu QHuang Y(2016)Convolutional neural random fields for action recognitionPattern Recognition10.1016/j.patcog.2016.03.01959:C(213-224)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1016/j.patcog.2016.03.019
Chen GXu RSrihari S(2016)Sequential Labeling with Online Deep Learning: Exploring Model InitializationMachine Learning and Knowledge Discovery in Databases10.1007/978-3-319-46227-1_48(772-788)Online publication date: 4-Sep-2016
https://doi.org/10.1007/978-3-319-46227-1_48
Srijith PBalamurugan PShevade S(2016)Gaussian Process Pseudo-Likelihood Models forźSequence LabelingEuropean Conference on Machine Learning and Knowledge Discovery in Databases - Volume 985110.1007/978-3-319-46128-1_14(215-231)Online publication date: 19-Sep-2016
https://dl.acm.org/doi/10.1007/978-3-319-46128-1_14
Fernandes EBrefeld UBlanco RAtserias J(2016)Using Wikipedia for Cross-Language Named Entity RecognitionBig Data Analytics in the Social and Ubiquitous Context10.1007/978-3-319-29009-6_1(1-25)Online publication date: 7-Jan-2016
https://doi.org/10.1007/978-3-319-29009-6_1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten