Skip to main content

Enhancing False-Sentence Pairs of BERT-Pair for Low-Frequency Aspect Category Detection

  • Conference paper
  • First Online:
Advances and Trends in Artificial Intelligence. Theory and Applications (IEA/AIE 2024)

Abstract

Aspect category detection (ACD), a task for detecting the aspect categories from texts, helps to extract the information customers need from review sentences automatically. BERT-pair was proposed, effectively solving this task by constructing auxiliary sentences from categories and classifying pairs of review and auxiliary sentences. In contrast, several product genres with multiple components include various low-frequency aspect categories in their reviews. Nevertheless, BERT-pair struggles to detect these categories; thus, the ACD performance for such products is poor. Moreover, to the best of our knowledge, no study on low-frequency category detection exists. Thus, we propose improved methods of BERT-pair to increase the performance for low-frequency categories. Our methods replace the review sentences in “false sentence pairs” that train BERT-pairs with similar sentences. This replacement enhances the diversity of sentences for low-frequency categories without additional training costs. Our experiment demonstrates that one of our methods increases the macro-F1 value for low-frequency categories by up to 0.11 points while maintaining the macro-F1 value for all categories.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://nlpaug.readthedocs.io/en/latest/augmenter/char/keyboard.html.

  2. 2.

    We conducted preliminary experiments using similar sentence generation methods, including AEDA [6], BT [4], DRAWS [8], PWSS [8], EDA [14], and KA. Consequently, we selected KA, which exhibited the best performance. In addition, we selected the LLM-based method inspired by its rich language generation capability.

  3. 3.

    https://alt.qcri.org/semeval2016/task5/.

  4. 4.

    https://huggingface.co.

  5. 5.

    KA-BERT-pair outperforms other data augmentation methods, not discussed in this paper owing to space limitations.

References

  1. Chebolu, S.U.S., et al.: Survey on aspect category detection. ACM Comput. Surv. 55(7, 132), 1–37 (2022)

    Google Scholar 

  2. Cheng, Z., et al.: Tell model where to attend: improving interpretability of aspect-based sentiment classification via small explanation annotations. In: Proceedings of the ICASSP 2023, pp. 1–5 (2023)

    Google Scholar 

  3. Devlin, J., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the NAACL-HLT 2019, pp. 4171–4186 (2019)

    Google Scholar 

  4. Edunov, S., et al.: Understanding back-translation at scale. In: Proceedings of the EMNLP 2018, pp. 489–500 (2018)

    Google Scholar 

  5. Hong, H., Song, J.: Utilizing BERT for detecting aspect categories on TABSA via adjusting self-attention among words. In: Proceedings of the ICHCI 2020, pp. 66–70 (2020)

    Google Scholar 

  6. Karimi, A., Rossi, L., Prati, A.: AEDA: an easier data augmentation technique for text classification. In: Findings of EMNLP 2021, pp. 2748–2754 (2021)

    Google Scholar 

  7. Ke, C., et al.: SimCPD: a simple framework for contrastive prompts of target-aspect-sentiment joint detection. Neural Comput. Appl. 35, 16577–16592 (2023)

    Article  Google Scholar 

  8. Li, G., et al.: Data augmentation for aspect-based sentiment analysis. Int. J. Mach. Learn. Cybern. 14, 125–133 (2023)

    Article  Google Scholar 

  9. Nooten, J.V., Daelemans, W.: Improving Dutch vaccine hesitancy monitoring via multi-label data augmentation with GPT-3.5. In: Proceedings of the WASSA 2023, pp. 251–270 (2023)

    Google Scholar 

  10. Okimura, I., et al.: On the impact of data augmentation on downstream performance in natural language processing. In: Proceedings of the insights 2022, pp. 88–93 (2022)

    Google Scholar 

  11. Pontiki, M., et al.: SemEval-2016 Task 5: aspect based sentiment analysis. In: Proceedings of the SemEval-2016, pp. 19–30 (2016)

    Google Scholar 

  12. Sun, C., Huang, L., Qiu, X.: Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of the NAACL-HLT 2019, pp. 380–385 (2019)

    Google Scholar 

  13. Wan, H., et al.: Target-aspect-sentiment joint detection for aspect-based sentiment analysis. In: Proceedings of the AAAI 2020, pp. 9122–9129 (2020)

    Google Scholar 

  14. Wei, J., et al.: EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In: Proceedings of the EMNLP-IJCNLP 2019, pp. 6382–6388 (2019)

    Google Scholar 

  15. Zhou, X., et al.: Dynamic multichannel fusion mechanism based on a graph attention network and BERT for aspect-based sentiment classification. Appl. Intell. 53, 6800–6813 (2023)

    Article  Google Scholar 

Download references

Acknowledgment

This work was supported in part by JSPS KAKENHI Grant Numbers JP22K18006, JP24K03052 and the Hibi Science Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Masato Kikuchi .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kikuchi, M., Anda, S., Ozono, T. (2024). Enhancing False-Sentence Pairs of BERT-Pair for Low-Frequency Aspect Category Detection. In: Fujita, H., Cimler, R., Hernandez-Matamoros, A., Ali, M. (eds) Advances and Trends in Artificial Intelligence. Theory and Applications. IEA/AIE 2024. Lecture Notes in Computer Science(), vol 14748. Springer, Singapore. https://doi.org/10.1007/978-981-97-4677-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-4677-4_12

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-4676-7

  • Online ISBN: 978-981-97-4677-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics