Skip to main content
SpringerLink
Log in

Hyperparameter tuning in convolutional neural networks for domain adaptation in sentiment classification (HTCNN-DASC)

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In-domain adaptation (DA), the knowledge trained in one domain, is used to test an unknown domain. Existing approaches use limited efforts on DA in sentiment classification (SC) using neural networks. The challenging task here is the dissimilarity in the semantic behavior across domains. In this paper, convolutional neural networks (CNNs) learn the knowledge of a particular domain using Doc2Vec feature representation which provides good performance for DA in SC for the target domain. Our empirical analysis with one-layer CNN exhibits significant change in the accuracy by tuning the hyperparameters involved with the CNN. This paper derives into a suitable CNN architecture accompanying hyperparameters which favor DA between different domains. Our empirical analysis with multi-domain dataset demonstrates that with suitable hyperparameters, CNN works well for DASC problems. The comparative study shows that CNN with Doc2Vec model provides a strong capability of learning large data representation semantically with other state-of-the-art methods for the DASC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://en.wikipedia.org/wiki/List_of_emoticons

  2. http://www.cs.jhu.edu/~mdredze/code.php

References

  • Blitzer J, Dredze M, Pereira F et al (2007) Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. ACL 7:440–447

    Google Scholar 

  • Bollegala D, Weir D, Carroll J (2013) Cross-domain sentiment classification using a sentiment sensitive thesaurus. IEEE Trans Knowl Data Eng 25:1719–1731

    Article  Google Scholar 

  • Dai W, Yang Q, Xue GR, Yu Y (2007) Boosting for transfer learning. In: Proceedings of the 24th international conference on machine learning. ACM, pp 193–200

  • Dai AM, Olah C, Le QV (2015) Document embedding with paragraph vectors. In: NIPS deep learning workshop

  • Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 513–520

  • He Y, Lin C, Alani H (2011) Automatically extracting polarity-bearing topics for cross-domain sentiment classification. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-Volume 1. Association for Computational Linguistics, pp 123–131

  • Jain V, Learned-Miller E (2011) Online domain adaptation of a pre-trained cascade of classifiers. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 577–584

  • Johnson R, Zhang T (2015) Effective use of word order for text categorization with convolutional neural networks. In: Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 103–112

  • Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Baltimore, Maryland, pp 655–665. http://www.aclweb.org/anthology/P/P14/P14-1062. Accessed 4 Dec 2016

  • Kandaswamy C, Silva LM, Alexandre LA, Santos JM, de Sá JM (2014) Improving deep neural network performance by reusing features trained with transductive transference. In: International conference on artificial neural networks. Springer, pp 265–272

  • Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp 1746–1751. http://aclweb.org/anthology/D/D14/D14-1181.pdf. Accessed 12 Feb 2017

  • Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 1188–1196

  • Li L, Qin B, Ren W, Liu T (2017) Document representation and feature combination for deceptive spam review detection. Neurocomputing 254:33–41

    Article  Google Scholar 

  • Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. In: Proceedings of the international conference on learning representations (ICLR) 2013

  • Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  • Pan W, Zhong E, Yang Q (2012) Transfer learning for text mining. In: Aggarwal C, Zhai C (eds) Mining text data. Springer, Boston, MA, pp 223–257

    Chapter  Google Scholar 

  • Rojas-Barahona LM (2016) Deep learning for sentiment analysis. Lang Linguist Compass 10(12):701–719

    Article  Google Scholar 

  • Rong X (2014) word2vec parameter learning explained. CoRR arxiv: abs/1411.2738

  • Rosenstein MT, Marx Z, Kaelbling LP, Dietterich TG (2005) To transfer or not to transfer. In: NIPS 2005 workshop on transfer learning, vol 898

  • Sanguansat P (2016) Paragraph2vec-based sentiment analysis on social media for business in Thailand. In: 2016 8th international conference on knowledge and smart technology (KST). IEEE, pp 175–178

  • Shen Y, He X, Gao J, Deng L, Mesnil G (2014) Learning semantic representations using convolutional neural networks for web search. In: Proceedings of the 23rd international conference on world wide web. ACM, pp 373–374

  • Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the international conference on learning representations (ICLR) 2015

  • Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92

    Article  Google Scholar 

  • Tang D, Qin B, Liu T (2015) Deep learning for sentiment analysis: successful approaches and future challenges. Wiley Interdiscip Rev Data Min Knowl Discov 5(6):292–303

    Article  Google Scholar 

  • Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: Proceedings of the 2016 conference on empirical methods in natural language processing. ACM, pp 214–224

  • Wei X, Lin H, Yu Y, Yang L (2017) Low-resource cross-domain product review sentiment classification based on a cnn with an auxiliary large-scale corpus. Algorithms 10(3):81

    Article  Google Scholar 

  • Whitehead M, Yaeger L (2009) Building a general purpose cross-domain sentiment mining model. In: 2009 WRI world congress on computer science and information engineering. IEEE, vol 4, pp 472–476

  • Wu H, Gu Y, Sun S, Gu X (2016) Aspect-based opinion summarization with convolutional neural networks. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 3157–3163

  • Wu F, Huang Y, Yuan Z (2017) Domain-specific sentiment classification via fusing sentiment knowledge from multiple sources. Inf Fusion 35:26–37

    Article  Google Scholar 

  • Zeng D, Liu K, Lai S, Zhou G, Zhao J et al (2014) Relation classification via convolutional deep neural network. In: COLING, pp 2335–2344

  • Zhang Y, Wallace B (2017) A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In: Proceedings of the 8th international joint conference on natural language processing, AFNLP, pp 253–263

  • Zhang S, Liu H, Yang L, Lin H (2015) A cross-domain sentiment classification method based on extraction of key sentiment sentence. In: Li J, Ji H, Zhao D, Feng Y (eds) Natural language processing and Chinese computing. Springer, Cham, pp 90–101

    Chapter  Google Scholar 

  • Zhou S, Chen Q, Wang X (2013) Active deep learning method for semi-supervised sentiment classification. Neurocomputing 120:536–546

    Article  Google Scholar 

  • Zhu E, Huang G, Mo B, Wu Q (2016) Features extraction based on neural network for cross-domain sentiment classification. In: International conference on database systems for advanced applications. Springer, pp 81–88

Download references

Acknowledgements

The authors thank the anonymous reviewers for their valuable suggestions and comments.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, A.V.C. College of Engineering, Mannampandal, Mayiladuthurai, 609 305, Tamilnadu, India

    K. Krishnakumari

  2. Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, Tiruchirappalli, 600 015, Tamilnadu, India

    E. Sivasankar & Sam Radhakrishnan

Authors
  1. K. Krishnakumari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. Sivasankar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sam Radhakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Krishnakumari.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krishnakumari, K., Sivasankar, E. & Radhakrishnan, S. Hyperparameter tuning in convolutional neural networks for domain adaptation in sentiment classification (HTCNN-DASC). Soft Comput 24, 3511–3527 (2020). https://doi.org/10.1007/s00500-019-04117-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04117-w

Keywords

Search

Navigation