Abstract
Context and motivation: Advances in Machine Learning (ML) and Deep Learning (DL) technologies have transformed the field of Natural Language Processing (NLP), making NLP more practical and accessible. Motivated by these exciting developments, Requirements Engineering (RE) researchers have been experimenting ML/DL based approaches for a range of RE tasks, such as requirements classification, requirements tracing, ambiguity detection, and modelling. Question/problem: Most of today’s ML/DL approaches are based on supervised learning techniques, meaning that they need to be trained using annotated datasets to learn how to assign a class label to examples from an application domain. This requirement poses an enormous challenge to RE researchers, as the lack of requirements datasets in general and annotated datasets in particular, makes it difficult for them to fully exploit the benefit of the advanced ML/DL technologies. Principal ideas/results: To address this challenge, this paper proposes a novel approach that employs the Zero-Shot Learning (ZSL) technique to perform requirements classification. We build several classification models using ZSL. We focus on the classification task because many RE tasks can be solved as classification problems by a large number of available ML/DL methods. In this preliminary study, we demonstrate our approach by classifying non-functional requirements (NFRs) into two categories: Usability and Security. ZSL supports learning without domain-specific training data, thus solving the lack of annotated datasets typical of RE. The study shows that our approach achieves an average of 82% recall and F-score. Contribution: This study demonstrates the potential of ZSL for requirements classification. The promising results of this study pave the way for further investigations and large-scale studies. An important implication is that it is possible to have very little or no training data to perform requirements classification. The proposed approach thus contributes to the solution of the long-standing problem of data shortage in RE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
BERT has two basic models: BERT\(_\text {base}\) and BERT\(_\text {large}\). BERT\(_\text {base}\) has 12 encoder layers whereas BERT\(_\text {large}\) has 24. Which BERT model to use depends on the application and BERT\(_\text {base}\) is usually sufficient for experiments, as it takes less time and resource to fine-tune comparing to BERT\(_\text {large}\).
- 2.
- 3.
The results are available at: https://github.com/waadalhoshan/ZSL4REQ.
References
Berry, D.M.: Empirical evaluation of tools for hairy requirements engineering tasks. Empir. Softw. Eng. 26(6), 1–77 (2021). https://doi.org/10.1007/s10664-021-09986-0
Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)
Dabrowski, J., Letier, E., Perini, A., Susi, A.: App review analysis for software engineering: a systematic literature review. University College London, Technical report (2020)
Dalpiaz, F., Dell’Anna, D., Aydemir, F.B., Çevikol, S.: Requirements classification with interpretable machine learning and dependency parsing. In: RE 2019, pp. 142–152. IEEE (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Ferrari, A., Dell’Orletta, F., Esuli, A., Gervasi, V., Gnesi, S.: Natural language requirements processing: a 4d vision. IEEE Softw. 34(6), 28–35 (2017)
Hey, T., Keim, J., Koziolek, A., Tichy, W.F.: NoRBERT: transfer learning for requirements classification. In: RE 2020, pp. 169–179. IEEE (2020)
Kurtanović, Z., Maalej, W.: Automatically classifying functional and non-functional requirements using supervised machine learning. In: RE 2017, pp. 490–495. IEEE (2017)
Liu, Y., et al.: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Maalej, W., Kurtanović, Z., Nabil, H., Stanik, C.: On the automatic classification of app reviews. Requirements Eng. 21(3), 311–331 (2016). https://doi.org/10.1007/s00766-016-0251-9
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
Pushp, P.K., Srivastava, M.M.: Train once, test anywhere: zero-shot learning for text classification. arXiv preprint arXiv:1712.05972 (2017)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training. Technical report, OpenAI (2018)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. arXiv preprint arXiv:1908.10084 (2019)
Romera-Paredes, B., Torr, P.: An embarrassingly simple approach to zero-shot learning. In: ICML 2015, pp. 2152–2161 (2015)
Ruder, S., Peters, M.E., Swayamdipta, S., Wolf, T.: Transfer learning in natural language processing. In: NACL 2019, pp. 15–18 (2019)
Vaswani, A., et al.: Attention is all you need. In: NeurIPS 2017, pp. 5998–6008 (2017)
Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., Zhou, M.: MiniLM: deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957 (2020)
Weiss, K., Khoshgoftaar, T.M., Wang, D.D.: A survey of transfer learning. J. Big Data 3(1), 1–40 (2016). https://doi.org/10.1186/s40537-016-0043-6
Wolf, T., Debut, L., Sanh, V., et al.: Transformers: state-of-the-art natural language processing. In: EMNLP 2020, pp. 38–45 (2020)
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. In: NeurIPS 2019, vol. 32 (2019)
Yin, W., Hay, J., Roth, D.: Benchmarking zero-shot text classification: datasets, evaluation and entailment approach. CoRR abs/1909.00161 (2019)
Zhao, L., et al.: Natural language processing for requirements engineering: a systematic mapping study. ACM Comput. Surv. 54(3), 55:1–55:41 (2021)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Alhoshan, W., Zhao, L., Ferrari, A., Letsholo, K.J. (2022). A Zero-Shot Learning Approach to Classifying Requirements: A Preliminary Study. In: Gervasi, V., Vogelsang, A. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2022. Lecture Notes in Computer Science, vol 13216. Springer, Cham. https://doi.org/10.1007/978-3-030-98464-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-98464-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98463-2
Online ISBN: 978-3-030-98464-9
eBook Packages: Computer ScienceComputer Science (R0)