Skip to main content
Springer Nature Link
Log in

SPACL: Shared-Private Architecture Based on Contrastive Learning for Multi-domain Text Classification

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13603))

Included in the following conference series:

  • 535 Accesses

Abstract

With the development of deep learning in recent years, text classification research has achieved remarkable results. However, text classification task often requires a large amount of annotated data, and data in different fields often force the model to learn different knowledge. It is often difficult for models to distinguish data labeled in different domains. Sometimes data from different domains can even damage the classification ability of the model and reduce the overall performance of the model. To address these issues, we propose a shared-private architecture based on contrastive learning for multi-domain text classification which can improve both the accuracy and robustness of classifiers. Extensive experiments are conducted on two public datasets. The results of experiments show that the our approach achieves the state-of-the-art performance in multi-domain text classification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Blitzer, J., Dredze, M., Pereira, F.: Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 440–447 (2007)

    Google Scholar 

  2. Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks. Advances in Neural Information Processing Systems 29 (2016)

    Google Scholar 

  3. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)

    Google Scholar 

  4. Chen, X., Cardie, C.: Multinomial adversarial networks for multi-domain text classification. arXiv preprint arXiv:1802.05694 (2018)

  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  6. Fang, H., Wang, S., Zhou, M., Ding, J., Xie, P.: Cert: contrastive self-supervised learning for language understanding. arXiv preprint arXiv:2005.12766 (2020)

  7. Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2030–2096 (2016)

    MathSciNet  Google Scholar 

  8. Gao, T., Yao, X., Chen, D.: Simcse: simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021)

  9. Giorgi, J., Nitski, O., Wang, B., Bader, G.: Declutr: deep contrastive learning for unsupervised textual representations. arXiv preprint arXiv:2006.03659 (2020)

  10. Hu, M., Wu, Y., Zhao, S., Guo, H., Cheng, R., Su, Z.: Domain-invariant feature distillation for cross-domain sentiment classification. arXiv preprint arXiv:1908.09122 (2019)

  11. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)

  12. Li, S., Zong, C.: Multi-domain sentiment classification. In: Proceedings of ACL-08: HLT, Short Papers, pp. 257–260 (2008)

    Google Scholar 

  13. Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101 (2016)

  14. Liu, P., Qiu, X., Huang, X.: Adversarial multi-task learning for text classification. arXiv preprint arXiv:1704.05742 (2017)

  15. Liu, X., Gao, J., He, X., Deng, L., Duh, K., Wang, Y.Y.: Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 912–921. Association for Computational Linguistics, Denver, Colorado (2015). https://doi.org/10.3115/v1/N15-1092, https://aclanthology.org/N15-1092/

  16. Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  17. Long, M., Cao, Z., Wang, J., Jordan, M.I.: Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems 31 (2018)

    Google Scholar 

  18. Meng, Y., Xiong, C., Bajaj, P., Bennett, P., Han, J., Song, X., et al.: Coco-lm: correcting and contrasting text sequences for language model pretraining. Advances in Neural Information Processing Systems 34 (2021)

    Google Scholar 

  19. Pan, S.J., Ni, X., Sun, J.T., Yang, Q., Chen, Z.: Cross-domain sentiment classification via spectral feature alignment. In: Proceedings of the 19th International Conference on World Wide Web, pp. 751–760 (2010)

    Google Scholar 

  20. Shen, J., Qu, Y., Zhang, W., Yu, Y.: Wasserstein distance guided representation learning for domain adaptation. In: Thirty-second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  21. Wang, C., Qiu, M., Huang, J., He, X.: Meta fine-tuning neural language models for multi-domain text mining. arXiv preprint arXiv:2003.13003 (2020)

  22. Wu, F., Huang, Y.: Collaborative multi-domain sentiment classification. In: 2015 IEEE International Conference on Data Mining, pp. 459–468. IEEE (2015)

    Google Scholar 

  23. Wu, Y., Guo, Y.: Dual adversarial co-learning for multi-domain text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6438–6445 (2020)

    Google Scholar 

  24. Wu, Y., Inkpen, D., El-Roby, A.: Conditional adversarial networks for multi-domain text classification. arXiv preprint arXiv:2102.10176 (2021)

  25. Wu, Y., Inkpen, D., El-Roby, A.: Mixup regularized adversarial networks for multi-domain text classification. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7733–7737. IEEE (2021)

    Google Scholar 

  26. Yang, Q., Shang, L.: Multi-task learning with bidirectional language models for text classification. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)

    Google Scholar 

Download references

Acknowledgement

This work has been supported by the Ministry of education of Humanities and Social Science project under Grant No. 19YJAZH128 and No. 20YJAZH118, the Science and Technology Plan Project of Guangzhou under Grant No. 202102080305.

Author information

Authors and Affiliations

  1. School of Cyber Security, Guangdong University of Foreign Studies, Guangzhou, 510006, China

    Guoding Xiong, Yongmei Zhou & Deheng Wang

  2. Guangzhou Key Laboratory of Multilingual Intelligent Processing, Guangdong University of Foreign Studies, Guangzhou, 510006, China

    Yongmei Zhou

  3. School of Computing, University of Leeds, Wood-house Lane, Leeds, West Yorkshire, LS2 9JT, UK

    Zhouhao Ouyang

Authors
  1. Guoding Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongmei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhouhao Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongmei Zhou .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Maosong Sun

  2. Tsinghua University, Beijing, China

    Yang Liu

  3. Harbin Institute of Technology, Harbin, China

    Wanxiang Che

  4. Chinese Academy of Sciences, Institute of Computing Technology, Beijing, China

    Yang Feng

  5. Fudan University, Shanghai, China

    Xipeng Qiu

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

  7. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Yubo Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xiong, G., Zhou, Y., Wang, D., Ouyang, Z. (2022). SPACL: Shared-Private Architecture Based on Contrastive Learning for Multi-domain Text Classification. In: Sun, M., et al. Chinese Computational Linguistics. CCL 2022. Lecture Notes in Computer Science(), vol 13603. Springer, Cham. https://doi.org/10.1007/978-3-031-18315-7_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-18315-7_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-18314-0

  • Online ISBN: 978-3-031-18315-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics