Skip to main content
SpringerLink
Log in

Deep domain adaptation with manifold aligned label transfer

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

We propose a novel deep learning domain adaptation method that performs transductive learning from the source domain to the target domain based on cluster matching between the source and target features. The proposed method combines Adaptive Batch Normalization and Locality Preserving Projection-based subspace alignment on deep features to produce a common feature space for label transfer. Adaptive Batch Normalization automatically conditions the features from the source/target domain by normalizing the activations in each layer of our network. Following Manifold Subspace Alignment, we cluster the data in each domain using Gaussian Mixture Model clustering in feature space. The clusters are matched between domains to transfer labels from the closest source cluster to each target cluster. The transfer labels are compared to the network prediction, and the samples with consistent labels are used to adapt the network on the target domain. The proposed manifold-guided label transfer method produces state-of-the-art results for deep adaptation on digit recognition datasets. Furthermore, we perform domain adaptation on remote sensing datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)

    Article  Google Scholar 

  2. Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., Krishnan, D.: Unsupervised pixel-level domain adaptation with generative adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 7 (2017)

  3. Dai, W., Xue, G.R., Yang, Q., Yu, Y.: Transferring naive Bayes classifiers for text classification. AAAI 7, 540–545 (2007)

    Google Scholar 

  4. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893. IEEE (2005)

  5. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009 (CVPR 2009), pp. 248–255. IEEE (2009)

  7. Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655 (2014)

  8. Elshamli, A., Taylor, G.W., Berg, A., Areibi, S.: Domain adaptation using representation learning for the classification of remote sensing images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(9), 4198–4209 (2017). https://doi.org/10.1109/JSTARS.2017.2711360

    Article  Google Scholar 

  9. Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 2960–2967. IEEE (2013)

  10. French, G., Mackiewicz, M., Fisher, M.: Self-Ensembling for Domain Adaptation. arXiv preprint arXiv:1706.05208v3 (2017)

  11. Ganin, Y., Lempitsky, V.: Unsupervised Domain Adaptation by Backpropagation. arXiv preprint arXiv:1409.7495 (2014)

  12. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2030–2096 (2016)

    MathSciNet  MATH  Google Scholar 

  13. Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation. In: European Conference on Computer Vision, pp. 597–613. Springer, Berlin (2016)

  14. Gong, B., Grauman, K., Sha, F.: Connecting the dots with landmarks: discriminatively learning domain-invariant features for unsupervised domain adaptation. In: International Conference on Machine Learning, pp. 222–230 (2013)

  15. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

  16. Gopalan, R., Li, R., Chellappa, R.: Domain adaptation for object recognition: an unsupervised approach. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 999–1006. IEEE (2011)

  17. Haeusser, P., Frerix, T., Mordvintsev, A., Cremers, D.: Associative domain adaptation. In: International Conference on Computer Vision (ICCV), vol. 2, p. 6 (2017)

  18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  19. Hoffman, J., Rodner, E., Donahue, J., Darrell, T., Saenko, K.: Efficient Learning of Domain-invariant Image Representations. arXiv preprint arXiv:1301.3224 (2013)

  20. Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, p. 3 (2017)

  21. Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)

    Article  Google Scholar 

  22. Krizhevsky, A., Hinton, G.: Learning Multiple Layers of Features from Tiny Images (2009)

  23. Kulis, B., Saenko, K., Darrell, T.: What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1785–1792. IEEE (2011)

  24. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  25. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  26. Lee, D.H.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML, vol. 3, p. 2 (2013)

  27. Li, H., Xu, Z., Taylor, G., Goldstein, T.: Visualizing the Loss Landscape of Neural Nets. arXiv preprint arXiv:1712.09913 (2017)

  28. Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting Batch Normalization for Practical Domain Adaptation. arXiv preprint arXiv:1603.04779 (2016)

  29. Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning Transferable Features with Deep Adaptation Networks. arXiv preprint arXiv:1502.02791 (2015)

  30. Lowe, D.G.: Object recognition from local scale-invariant features. In: The Proceedings of the Seventh IEEE International Conference on Computer vision, 1999, vol. 2, pp. 1150–1157. IEEE (1999)

  31. Maaten, Lvd, Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  32. Minnehan, B., Savakis, A.: Manifold guided label transfer for deep domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 65–73 (2017)

  33. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning, vol. 2011, pp. 1–9 (2011)

  34. Ranjan, V., Harit, G., Jawahar, C.: Domain adaptation by aligning locality preserving subspaces. In: 2015 Eighth International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–6. IEEE (2015)

  35. Russo, P., Carlucci, F.M., Tommasi, T., Caputo, B.: From Source to Target and Back: Symmetric Bi-directional Adaptive Gan. arXiv preprint arXiv:1705.08824 (2017)

  36. Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: European Conference on Computer Vision, pp. 213–226. Springer, Berlin (2010)

  37. Saito, K., Ushiku, Y., Harada, T.: Asymmetric Tri-training for Unsupervised Domain Adaptation. arXiv preprint arXiv:1702.08400 (2017)

  38. Samat, A., Persello, C., Gamba, P., Liu, S., Abuduwaili, J., Li, E.: Supervised and semi-supervised multi-view canonical correlation analysis ensemble for heterogeneous domain adaptation in remote sensing image classification. Remote Sens. 9(337) (2017). https://doi.org/10.3390/rs9040337. http://www.mdpi.com/2072-4292/9/4/337

  39. Sankaranarayanan, S., Balaji, Y., Castillo, C.D., Chellappa, R.: Generate to Adapt: Aligning Domains Using Generative Adversarial Networks. arXiv preprint arXiv:1704.01705 (2017)

  40. Sener, O., Song, H.O., Saxena, A., Savarese, S.: Learning transferrable representations for unsupervised domain adaptation. In: Advances in Neural Information Processing Systems, pp. 2110–2118 (2016)

  41. Sener, O., Song, H.O., Saxena, A., Savarese, S.: Unsupervised Transductive Domain Adaptation. arXiv preprint arXiv:1602.03534 (2016)

  42. Shekhar, S., Patel, V.M., Nguyen, H.V., Chellappa, R.: Generalized domain-adaptive dictionaries. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 361–368. IEEE (2013)

  43. Shi, Q., Du, B., Zhang, L.: Domain adaptation for remote sensing image classification: a low-rank reconstruction and instance weighting label propagation inspired algorithm. IEEE Trans. Geosci. Remote Sens. 53(10), 5677–5689 (2015). https://doi.org/10.1109/TGRS.2015.2427791

    Article  Google Scholar 

  44. Shi, Y., Sha, F.: Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation. arXiv preprint arXiv:1206.6438 (2012)

  45. Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. AAAI 6, 2058–2065 (2016)

    Google Scholar 

  46. Sun, B., Saenko, K.: Deep coral: Correlation alignment for deep domain adaptation. In: European Conference on Computer Vision, pp. 443–450. Springer, Berlin (2016)

  47. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1521–1528. IEEE (2011)

  48. Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. Comput. Vis. Pattern Recognit. 1, 2962–2971 (2017)

    Google Scholar 

  49. Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep Domain Confusion: Maximizing for Domain Invariance. arXiv preprint arXiv:1412.3474 (2014)

  50. Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: European Conference on Computer Vision (ECCV) (2016)

  51. Xia, G.S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., Lu, X.: Aid: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans. Geosci. Remote Sens. 55(7), 3965–3981 (2017)

    Article  Google Scholar 

  52. Yang, Y., Newsam, S.: Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 270–279. ACM (2010)

  53. Zhang, X., Yu, F.X., Chang, S.F., Wang, S.: Deep Transfer Network: Unsupervised Domain Adaptation. arXiv preprint arXiv:1503.00591 (2015)

  54. Zheng, X., Cai, D., He, X., Ma, W.Y., Lin, X.: Locality preserving clustering for image database. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 885–891. ACM (2004)

Download references

Acknowledgements

This research was supported in part by a grant from the AFOSR Dynamic Data Driven Applications Systems (DDDAS) program.

Author information

Authors and Affiliations

  1. Rochester Institute of Technology, 83 Lomb Memorial Drive, Rochester, NY, 14623-5603, USA

    Breton Minnehan & Andreas Savakis

Authors
  1. Breton Minnehan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Savakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Breton Minnehan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Minnehan, B., Savakis, A. Deep domain adaptation with manifold aligned label transfer. Machine Vision and Applications 30, 473–485 (2019). https://doi.org/10.1007/s00138-019-01003-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-019-01003-1

Keywords

Search

Navigation