skip to main content
10.1145/1557019.1557130acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Cross domain distribution adaptation via kernel mapping

Published: 28 June 2009 Publication History

Abstract

When labeled examples are limited and difficult to obtain, transfer learning employs knowledge from a source domain to improve learning accuracy in the target domain. However, the assumption made by existing approaches, that the marginal and conditional probabilities are directly related between source and target domains, has limited applicability in either the original space or its linear transformations. To solve this problem, we propose an adaptive kernel approach that maps the marginal distribution of target-domain and source-domain data into a common kernel space, and utilize a sample selection strategy to draw conditional probabilities between the two domains closer. We formally show that under the kernel-mapping space, the difference in distributions between the two domains is bounded; and the prediction error of the proposed approach can also be bounded. Experimental results demonstrate that the proposed method outperforms both traditional inductive classifiers and the state-of-the-art boosting-based transfer algorithms on most domains, including text categorization and web page ratings. In particular, it can achieve around 10% higher accuracy than other approaches for the text categorization problem. The source code and datasets are available from the authors.

Supplementary Material

JPG File (p1027-zhong.jpg)
MP4 File (p1027-zhong.mp4)

References

[1]
M. Amini, F. Laviolette, and N. Usunier. A transductive bound for the voted classifier with an application to semi-supervised learning. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21. 2009.
[2]
A. Asuncion and D. J. Newman. UCI machine learning repository, 2007. http://www.ics.uci.edu/mlearn/MLRepository.html.
[3]
G. Baudat and F. Anouar. Generalized discriminant analysis using a kernel approach. Neural Comput., 12(10):2385--2404, 2000.
[4]
M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399--2434, 2006.
[5]
S. Bickel, M. Brlzckner, and T. Scheer. Discriminative learning for diering training and test distributions. In Z. Ghahramani, editor, ICML, volume 227 of ACM International Conference Proceeding Series, pages 81--88. ACM, 2007.
[6]
J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wortman. Learning bounds for domain adaptation. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20. 2008.
[7]
F. Cucker and S. Smale. Best choices for regularization parameters in learning theory: On the bias-variance problem. Foundations of Computational Mathematics, 2(4):413--428, 2002.
[8]
W. Dai, Q. Yang, G.-R. Xue, and Y. Yu. Boosting for transfer learning. In ICML '07: Proceedings of the 24th international conference on Machine learning, pages 193--200, New York, NY, USA, 2007. ACM.
[9]
D. Davidov, E. Gabrilovich, and S. Markovitch. Parameterized generation of labeled datasets for text categorization based on a hierarchical directory. In Proceedings of The 27th Annual International ACM SIGIR Conference, pages 250--257, Sheffield, UK, 2004. ACM Press.
[10]
I. Davidson and W. Fan. When efficient model averaging out-performs boosting and bagging. In Knowledge Discovery in Databases: PKDD 2006, 10th European Conference on Principles and Practice of Knowledge Discovery in Databases, Berlin, Germany, September 18-22, 2006, Proceedings, pages 478--486. Springer, 2006.
[11]
W. Fan and I. Davidson. On sample selection bias and its efficient correction via model averaging and unlabeled examples. In Proceedings of the Seventh SIAM International Conference on Data Mining, SDM 2007, Minneapolis, Minnesota, USA, 2007. SIAM.
[12]
J. Gao, W. Fan, J. Jiang, and J. Han. Knowledge transfer via multiple model local structure mapping. In Y. Li, B. Liu, and S. Sarawagi, editors, Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, August 24-27, 2008, pages 283--291. ACM, 2008.
[13]
S. Y. Huang and C. R. Hwang. Kernel fisher's discriminant analysis in gaussian reproducing kernel hilbert space. Technical report, Institute of Statistical Science, Academia Sinica, Taiwan, 2005.
[14]
S. J. Pan, J. T. Kwok, and Q. Yang. Transfer learning via dimensionality reduction. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008, pages 677--682. AAAI Press, 2008.
[15]
S. J. Pan and Q. Yang. A survey on transfer learning. Technical Report HKUST-CS08-08, Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China, November 2008.
[16]
J. Ren, X. Shi, W. Fan, and P. S. Yu. Type-independent correction of sample selection bias via structural discovery and re-balancing. In Proceedings of the Eighth SIAM International Conference on Data Mining, SDM 2008, pages 565--576, Atlanta, Georgia, USA, 2008. SIAM.
[17]
U. Rückert and S. Kramer. Kernel-based inductive transfer. In Machine Learning and Knowledge Discovery in Databases, European Conference, ECML/PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II, pages 220--233, 2008.
[18]
S. Satpal and S. Sarawagi. Domain adaptation of conditional probability models via feature subsetting. In J. N. Kok, J. Koronacki, R. L. de Mlcntaras, S. Matwin, D. Mladenic, and A. Skowron, editors, PKDD, volume 4702 of Lecture Notes in Computer Science, pages 224--235. Springer, 2007.
[19]
B. Schölkopf, R. Herbrich, and A. J. Smola. A generalized representer theorem. In COLT '01/EuroCOLT '01: Proceedings of the 14th Annual Conference on Computational Learning Theory and and 5th European Conference on Computational Learning Theory, pages 416--426, London, UK, 2001. Springer-Verlag.
[20]
Z. Wang, Y. Song, and C. Zhang. Transferred dimensionality reduction. In Machine Learning and Knowledge Discovery in Databases, European Conference, ECML/PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II, pages 550--565, 2008.
[21]
I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems). Morgan Kaufmann, June 2005.
[22]
K. Yamazaki, M. Kawanabe, S. Watanabe, M. Sugiyama, and K.-R. Müller. Asymptotic bayesian generalization error when training and test distributions are different. In ICML '07: Proceedings of the 24th international conference on Machine learning, pages 1079--1086, New York, NY, USA, 2007. ACM
[23]
P. Zhang, J. Peng, and N. Riedel. Discriminant analysis: A unified approach. In ICDM '05: Proceedings of the Fifth IEEE International Conference on Data Mining, pages 514--521, Washington, DC, USA, 2005. IEEE Computer Society.
[24]
X. Zhu. Semi-supervised learning with graphs. PhD thesis, Dept. of Computer Science, University of Carnegie Mellon, 2005.

Cited By

View all
  • (2025)Self-distillation-based domain exploration for source speaker verification under spoofed speech from unknown voice conversionSpeech Communication10.1016/j.specom.2024.103153167(103153)Online publication date: Feb-2025
  • (2024)Transfer Adaptation Learning: A Decade SurveyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.318332635:1(23-44)Online publication date: Jan-2024
  • (2024)Enhancing Information Extraction from Low-sample Materials Science Literature by Transfer Learning2024 10th International Conference on Big Data and Information Analytics (BigDIA)10.1109/BigDIA63733.2024.10808706(736-741)Online publication date: 25-Oct-2024
  • Show More Cited By

Index Terms

  1. Cross domain distribution adaptation via kernel mapping

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
    June 2009
    1426 pages
    ISBN:9781605584959
    DOI:10.1145/1557019
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 June 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. domain transfer
    2. ensemble
    3. generalization bound
    4. kernel

    Qualifiers

    • Research-article

    Conference

    KDD09

    Acceptance Rates

    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)17
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Self-distillation-based domain exploration for source speaker verification under spoofed speech from unknown voice conversionSpeech Communication10.1016/j.specom.2024.103153167(103153)Online publication date: Feb-2025
    • (2024)Transfer Adaptation Learning: A Decade SurveyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.318332635:1(23-44)Online publication date: Jan-2024
    • (2024)Enhancing Information Extraction from Low-sample Materials Science Literature by Transfer Learning2024 10th International Conference on Big Data and Information Analytics (BigDIA)10.1109/BigDIA63733.2024.10808706(736-741)Online publication date: 25-Oct-2024
    • (2024)Heterogeneous transfer learning: recent developments, applications, and challengesMultimedia Tools and Applications10.1007/s11042-024-18352-383:27(69759-69795)Online publication date: 2-Feb-2024
    • (2023)ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series DataACM Transactions on Knowledge Discovery from Data10.1145/358793717:8(1-18)Online publication date: 12-May-2023
    • (2023)Multi-objective dynamic distribution adaptation with instance reweighting for transfer feature learningKnowledge-Based Systems10.1016/j.knosys.2023.110303263:COnline publication date: 5-Mar-2023
    • (2023)Unsupervised domain adaptation without source data for estimating occupancy and recognizing activities in smart buildingsEnergy and Buildings10.1016/j.enbuild.2023.113808(113808)Online publication date: Dec-2023
    • (2023)Instance-representation transfer method based on joint distribution and deep adaptation for EEG emotion recognitionMedical & Biological Engineering & Computing10.1007/s11517-023-02956-262:2(479-493)Online publication date: 2-Nov-2023
    • (2023)Boost two-view learning-based method for label proportions problemApplied Intelligence10.1007/s10489-023-04643-z53:19(21984-22001)Online publication date: 18-Jun-2023
    • (2023)Prototype-based semantic consistency learning for unsupervised 2D image-based 3D shape retrievalMultimedia Systems10.1007/s00530-023-01086-x29:4(1995-2007)Online publication date: 11-Apr-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media