Skip to main content
SpringerLink
Log in
Book cover

European Conference on Computer Vision

ECCV 2022: Computer Vision – ECCV 2022 pp 200–217Cite as

Incomplete Multi-view Domain Adaptation via Channel Enhancement and Knowledge Transfer

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13694))

Abstract

Unsupervised domain adaptation (UDA) borrows well-labeled source knowledge to solve the specific task on unlabeled target domain with the assumption that both domains are from a single sensor, e.g., RGB or depth images. To boost model performance, multiple sensors are deployemd on new-produced devices like autonomous vehicles to benefit from enriched information. However, the model trained with multi-view data difficultly becomes compatible with conventional devices only with a single sensor. This scenario is defined as incomplete multi-view domain adaptation (IMVDA), which considers that the source domain consists of multi-view data while the target domain only includes single-view instances. To overcome this practical demand, this paper proposes a novel Channel Enhancement and Knowledge Transfer (CEKT) framework with two modules. Concretely, the source channel enhancement module distinguishes view-common from view-specific channels and explores channel similarity to magnify the representation of important channels. Moreover, the adaptive knowledge transfer module attempts to enhance target representation towards multi-view semantic through implicit missing view recovery and adaptive cross-domain alignment. Extensive experimental results illustrate the effectiveness of our method in solving the IMVDA challenge.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    This paper considers the case that the source domain contains two views while target domain includes only single view.

References

  1. Aljundi, R., Emonet, R., Muselet, D., Sebban, M.: Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 56–63 (2015)

    Google Scholar 

  2. Baktashmotlagh, M., Harandi, M.T., Lovell, B.C., Salzmann, M.: Unsupervised domain adaptation by domain invariant projection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 769–776 (2013)

    Google Scholar 

  3. Ben-David, S., Blitzer, J., Crammer, K., Pereira, F., et al.: Analysis of representations for domain adaptation. Adv. Neural Inf. Process. Syst. 19, 137 (2007)

    Google Scholar 

  4. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  5. Ding, Z., Fu, Y.: Low-rank common subspace for multi-view learning. In: 2014 IEEE international conference on Data Mining, pp. 110–119. IEEE (2014)

    Google Scholar 

  6. Ding, Z., Li, S., Shao, M., Fu, Y.: Graph adaptive knowledge transfer for unsupervised domain adaptation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 37–52 (2018)

    Google Scholar 

  7. Du, Z., Li, J., Su, H., Zhu, L., Lu, K.: Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2021)

    Google Scholar 

  8. Elbrächter, D., Perekrestenko, D., Grohs, P., Bölcskei, H.: Deep neural network approximation theory. IEEE Trans. Inf. Theory 67(5), 2581–2623 (2021)

    Article  MathSciNet  Google Scholar 

  9. Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2960–2967 (2013)

    Google Scholar 

  10. Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2030–2096 (2016)

    MathSciNet  Google Scholar 

  11. Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset (2007)

    Google Scholar 

  12. Guan, D., Huang, J., Lu, S., Xiao, A.: Scale variance minimization for unsupervised domain adaptation in image segmentation. Pattern Recognit. 112, 107764 (2021)

    Article  Google Scholar 

  13. Guan, D., Huang, J., Xiao, A., Lu, S., Cao, Y.: Uncertainty-aware unsupervised domain adaptation in object detection. IEEE Trans. Multimedia 24, 2502–2514 (2021)

    Article  Google Scholar 

  14. He, J., Jia, X., Chen, S., Liu, J.: Multi-source domain adaptation with collaborative learning for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11008–11017 (2021)

    Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  16. He, Y., Tian, Y., Liu, D.: Multi-view transfer learning with privileged learning framework. Neurocomputing 335, 131–142 (2019)

    Article  Google Scholar 

  17. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

    Google Scholar 

  18. Janoch, A., et al.: A Category-Level 3D Object dataset: putting the Kinect to Work. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds.) Consumer Depth Cameras for Computer Vision. Advances in Computer Vision and Pattern Recognition, pp 141–165. Springer, London (2013). https://doi.org/10.1007/978-1-4471-4640-7_8

  19. Jing, T., Liu, H., Ding, Z.: Towards novel target discovery through open-set domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9322–9331 (2021)

    Google Scholar 

  20. Jing, T., Xia, H., Hamm, J., Ding, Z.: Augmented multi-modality fusion for generalized zero-shot sketch-based visual retrieval. IEEE Trans. Image Process. 31, 3657–3668 (2022)

    Article  Google Scholar 

  21. Kang, G., Jiang, L., Yang, Y., Hauptmann, A.G.: Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4893–4902 (2019)

    Google Scholar 

  22. Kumar, A., Rai, P., Daume, H.: Co-regularized multi-view spectral clustering. Adv. Neural Inf. Process. Syst. 24, 1413–1421 (2011)

    Google Scholar 

  23. Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view rgb-d object dataset. In: 2011 IEEE International Conference on Robotics and Automation, pp. 1817–1824. IEEE (2011)

    Google Scholar 

  24. Li, J., Li, Z., Lu, G., Xu, Y., Zhang, B., Zhang, D.: Asymmetric gaussian process multi-view learning for visual classification. Inf. Fusion 65, 108–118 (2021)

    Article  Google Scholar 

  25. Li, R., Jiao, Q., Cao, W., Wong, H.S., Wu, S.: Model adaptation: unsupervised domain adaptation without source data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9641–9650 (2020)

    Google Scholar 

  26. Li, Y., Yuan, L., Chen, Y., Wang, P., Vasconcelos, N.: Dynamic transfer for multi-source domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10998–11007 (2021)

    Google Scholar 

  27. Liang, J., Hu, D., Feng, J.: Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In: International Conference on Machine Learning, pp. 6028–6039. PMLR (2020)

    Google Scholar 

  28. Liu, B., Chen, X., Xiao, Y., Li, W., Liu, L., Liu, C.: An efficient dictionary-based multi-view learning method. Inf. Sci. 576, 157–172 (2021)

    Article  MathSciNet  Google Scholar 

  29. Liu, X., et al.: Adversarial unsupervised domain adaptation with conditional and label shift: infer, align and iterate. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10367–10376 (2021)

    Google Scholar 

  30. Liu, Y., Wang, L., Bai, Y., Qin, C., Ding, Z., Fu, Y.: Generative view-correlation adaptation for semi-supervised multi-view learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 318–334. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58568-6_19

    Chapter  Google Scholar 

  31. Long, M., Cao, Z., Wang, J., Jordan, M.I.: Conditional adversarial domain adaptation. arXiv preprint arXiv:1705.10667 (2017)

  32. Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. In: International Conference on Machine Learning, pp. 2208–2217. PMLR (2017)

    Google Scholar 

  33. Lu, X., Wang, W., Danelljan, M., Zhou, T., Shen, J., Van Gool, L.: Video object segmentation with episodic graph memory networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12348, pp. 661–679. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58580-8_39

    Chapter  Google Scholar 

  34. Lu, Z., Yang, Y., Zhu, X., Liu, C., Song, Y.Z., Xiang, T.: Stochastic classifiers for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9111–9120 (2020)

    Google Scholar 

  35. Na, J., Jung, H., Chang, H.J., Hwang, W.: Fixbi: bridging domain spaces for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1094–1103 (2021)

    Google Scholar 

  36. Pan, Y., Yao, T., Li, Y., Wang, Y., Ngo, C.W., Mei, T.: Transferrable prototypical networks for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2239–2247 (2019)

    Google Scholar 

  37. Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., Wang, B.: Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1406–1415 (2019)

    Google Scholar 

  38. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  39. Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 213–226. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_16

    Chapter  Google Scholar 

  40. Tang, H., Chen, K., Jia, K.: Unsupervised domain adaptation via structurally regularized deep clustering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8725–8735 (2020)

    Google Scholar 

  41. Venkateswara, H., Eusebio, J., Chakraborty, S., Panchanathan, S.: Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5018–5027 (2017)

    Google Scholar 

  42. Wang, M., et al.: Interbn: channel fusion for adversarial unsupervised domain adaptation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3691–3700 (2021)

    Google Scholar 

  43. Wang, Q., Cheng, J., Gao, Q., Zhao, G., Jiao, L.: Deep multi-view subspace clustering with unified and discriminative learning. IEEE Trans. Multimedia 23, 3483–3493 (2020)

    Article  Google Scholar 

  44. Xia, H., Ding, Z.: Structure preserving generative cross-domain learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4364–4373 (2020)

    Google Scholar 

  45. Xia, H., Ding, Z.: Cross-domain collaborative normalization via structural knowledge. In: AAAI 2022 (2022)

    Google Scholar 

  46. Xia, H., Jing, T., Ding, Z.: Maximum structural generation discrepancy for unsupervised domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. (2022)

    Google Scholar 

  47. Xu, H., Zhang, X., Xia, W., Gao, Q., Gao, X.: Low-rank tensor constrained co-regularized multi-view spectral clustering. Neural Networks 132, 245–252 (2020)

    Article  Google Scholar 

  48. Zhang, C., Cui, Y., Han, Z., Zhou, J.T., Fu, H., Hu, Q.: Deep partial multi-view learning. IEEE Trans. Pattern Anal. Mach. Intell. 44, 2402–2415 (2020)

    Google Scholar 

  49. Zhang, C., Cai, Y., Lin, G., Shen, C.: Deepemd: few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12203–12213 (2020)

    Google Scholar 

  50. Zhang, D., et al.: Direct quantification of coronary artery stenosis through hierarchical attentive multi-view learning. IEEE Trans. Med. Imaging 39(12), 4322–4334 (2020)

    Article  Google Scholar 

  51. Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: which helps face recognition? In: 2011 International Conference on Computer Vision, pp. 471–478. IEEE (2011)

    Google Scholar 

  52. Zhao, J., Xie, X., Xu, X., Sun, S.: Multi-view learning overview: recent progress and new challenges. Inf. Fusion 38, 43–54 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Tulane University, New Orleans, USA

    Haifeng Xia & Zhengming Ding

  2. Mitsubishi Electric Research Laboratories, Cambridge, USA

    Pu Wang

Authors
  1. Haifeng Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengming Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haifeng Xia .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xia, H., Wang, P., Ding, Z. (2022). Incomplete Multi-view Domain Adaptation via Channel Enhancement and Knowledge Transfer. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13694. Springer, Cham. https://doi.org/10.1007/978-3-031-19830-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19830-4_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19829-8

  • Online ISBN: 978-3-031-19830-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation