Abstract
We propose a pipeline for transductive transfer learning and demonstrate it in computer vision tasks. In pattern classification, methods for transductive transfer learning (also known as unsupervised domain adaptation) are designed to cope with cases in which one cannot assume that training and test sets are sampled from the same distribution, i.e., they are from different domains. However, some unlabelled samples that belong to the same domain as the test set (i.e. the target domain) are available, enabling the learner to adapt its parameters. We approach this problem by combining three methods that transform the feature space. The first finds a lower dimensional space that is shared between source and target domains. The second uses local transformations applied to each source sample to further increase the similarity between the marginal distributions of the datasets. The third applies one transformation per class label, aiming to increase the similarity between the posterior probability of samples in the source and target sets. We show that this combination leads to an improvement over the state-of-the-art in cross-domain image classification datasets, using raw images or basic features and a simple one-nearest-neighbour classifier.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In Fig. 1, the feature space is visualised in 2D using PCA projection and only two classes are shown, but the MMD computation was done on a higher dimensional space on samples from 10 classes. For these reasons it may not be easy to see that the means of source and target samples became closer after MMD.
- 2.
References
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)
Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2011)
FarajiDavar, N., deCampos, T., Kittler, J.: Adaptive transductive transfer machines. In: Proceedings of the British Machine Vision Conference (BMVC), Nottingham (2014)
Cortes, C., Mohri, M., Riley, M.D., Rostamizadeh, A.: Sample selection bias correction theory. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds.) ALT 2008. LNCS (LNAI), vol. 5254, pp. 38–53. Springer, Heidelberg (2008)
Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., Schölkopf, B.: Covariate shift by kernel mean matching. Dataset Shift Mach. Learn. 3, 131–160 (2009)
Joachims, T.: Transductive inference for text classification using support vector machines. In: Proceedings of the International Conference Machine Learning, ICML, San Francisco, CA, USA, pp. 200–209 (1999)
Yang, J., Yan, R., Hauptmann, A.G.: Cross-domain video concept detection using adaptive. In: Proceedings of the 15th International Conference on Multimedia (2007). doi:10.1145/1291233.1291276
Gopalan, R., Li, R., Chellappa, R.: Unsupervised adaptation across domain shifts by generating intermediate data representations. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 36(11), 2288–2302 (2014). doi:10.1109/TPAMI.2013.249
Chu, W.S., De la Torre, F., Cohn, J.F.: Selective transfer machine for personalized facial action unit detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2013)
Dai, W., Chen, Y., Xue, G., Yang, Q., Yu, Y.: Translated learning: transfer learning across different feature spaces. In: Neural Information Processing Systems, pp. 353–360 (2008)
Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H., Schalkopf, B., Smola, A.J.: Integrating structured biological data by kernel maximum mean discrepancy. In: Proceedings of the International Conference Intelligent Systems for Molecular Biology (2006)
Long, M., Wang, J., Ding, G., Yu, P.: Transfer learning with joint distribution adaptation. In: Proceedings of the International Conference on Computer Vision, ICCV (2013)
Aytar, Y., Zisserman, A.: Tabula rasa: model transfer for object category detection. In: Proceedings of the International Conference on Computer Vision, ICCV (2011)
Gretton, A., Borgwardt, K., Rasch, M., Scholkopf, B., Smola, A.: A kernel method for the two sample problem. In: Proceedings of the Neural Information Processing Systems, NIPS, pp. 513–520. MIT Press (2007)
Sun, Q., Chattopadhyay, R., Panchanathan, S., Ye, J.: A two-stage weighting framework for multi-source domain adaptation. In: Proceedings of the Neural Information Processing Systems, NIPS, pp. 505–513 (2011)
Arnold, A., Nallapati, R., Cohen, W.W.: A comparative study of methods for transductive transfer learning. In: Proceedings of the Seventh IEEE International Conference on Data Mining Workshops, ICDMW, pp. 77–82. IEEE Computer Society, Washington (2007)
FarajiDavar, N., deCampos, T., Kittler, J., Yan, F.: Transductive transfer learning for action recognition in tennis games. In: VECTaR Workshop, in Conjunction with ICCV (2011)
Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence, pp. 1187–1192. Morgan Kaufmann Publishers Inc., San Francisco (2009)
Reynolds, D.A., Quatieri, T.F., Dunn, R.B.: Speaker verification using adapted gaussian mixture models. Digit. Sig. Process. 10, 19–41 (2000)
Chen, M., Weinberger, K.Q., Blitzer, J.: Co-training for domain adaptation. In: Proceedings of the Neural Information Processing Systems, NIPS, pp. 2456–2464 (2011)
Quanz, B., Huan, J., Mishra, M.: Knowledge transfer with low-quality data: a feature extraction issue. In: Abiteboul, S., Bolhm, K., Koch, C., Tan, K. (eds.) Proceedings of the 27th International Conference on Data Engineering (ICDE), pp. 769–779. IEEE Computer Society, Hannover, Germany (2011)
Zhong, E., Fan, W., Peng, J., Zhang, K., Ren, J., Turaga, D.S., Verscheure, O.: Cross domain distribution adaptation via kernel mapping. In: International Conference on Knowledge Discovery and Data mining, KDD, pp. 1027–1036. ACM (2009)
Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Plan. Infer. 90, 227–244 (2000)
Cun, Y.L., Boser, B., Denker, J.S., Howard, R.E., Habbard, W., Jackel, L.D., Henderson, D.: Handwritten digit recognition with a back-propagation network. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 2, pp. 396–404. Morgan Kaufmann Publishers Inc., San Francisco (1990)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia university image library COIL-20 (1996). http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php (retrieved 30 June 2014)
Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2066–2073 (2012)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110, 346–359 (2008)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the Neural Information Processing Systems, NIPS, pp. 1106–1114 (2012)
Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22, 199–210 (2011)
Si, S., Tao, D., Geng, B.: Bregman divergence-based regularization for transfer subspace learning. IEEE Trans. Knowl. Data Eng. 22, 929–942 (2010)
Gopalan, R., Li, R., Chellappa, R.: Domain adaptation for object recognition: an unsupervised approach. In: Metaxas, D.N., Quan, L., Sanfeliu, A., Gool, L.J.V. (eds.) Proceedings of the International Conference on Computer Vision, ICCV, pp. 999–1006 (2011)
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: DeCAF: A deep convolutional activation feature for generic visual recognition. Technical report CoRR arXiv:1310.1531, Cornell University Library (2013)
Duan, L., Tsang, I.W., Xu, D., Chua, T.: Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML, pp. 289–296. ACM, New York (2009)
Chen, L., Li, W., Xu, D.: Recognizing RGB images by learning from RGB-D data. In: IEEE International Conference on Computer Vision and Pattern Recognition, CVPR (2014)
Gao, J., Fan, W., Jiang, J., Han, J.: Knowledge transfer via multiple model local structure mapping. In: Knowledge Discovery and Data Mining, pp. 283–291 (2008)
Acknowledgements
We are grateful for the support of EPSRC/dstl contract EP/K014307/1 (Signal processing in a network battlespace) and EPSRC project S3A, EP/L000539/1. During part of the development of this work, TdC had been working in Neil Lawrence’s group at the University of Sheffield.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Farajidavar, N., de Campos, T., Kittler, J. (2015). Transductive Transfer Machine. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9005. Springer, Cham. https://doi.org/10.1007/978-3-319-16811-1_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-16811-1_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16810-4
Online ISBN: 978-3-319-16811-1
eBook Packages: Computer ScienceComputer Science (R0)