Abstract
Requirements are usually written in natural language and evolve continuously during the process of software development, which involves a large number of stakeholders. Stakeholders with diverse backgrounds and skills might refer to the same real-world entity with different linguistic expressions in the natural-language requirements, resulting in requirement inconsistency. We define this phenomenon as Entity Coreference (EC) in the Requirement Engineering (RE) area. It can lead to misconception about technical terminologies, and harm the readability and long-term maintainability of the requirements. In this paper, we propose a DEEP context-wise method for entity COREFerence detection, named DeepCoref. First, we truncate corresponding contexts surrounding entities. Then, we construct a deep context-wise neural network for coreference classification. The network consists of one fine-tuning BERT model for context representation, a Word2Vec-based network for entity representation, and a multi-layer perceptron in the end to fuse and make a trade-off between two representations. Finally, we cluster and normalize coreferent entities. We evaluate our method, respectively, on coreference classification and clustering with 1853 industry data on 21 projects. The former evaluation shows that DeepCoref outperforms three baselines with average precision and recall of 96.10% and 96.06%, respectively. The latter evaluation on six metrics shows that DeepCoref can cluster coreferent entities more accurately. We also conduct ablation experiments with three variants to demonstrate the performance enhancement brought by different components of neural network designed for coreference classification.
Similar content being viewed by others
Notes
Note that entities in this research are ready-made, and entity extraction is out of the scope of this research.
It is one of the world’s top five hundred commercial banks.
References
Arora C, Sabetzadeh M, Briand LC, Zimmer F (2017) Automated extraction and clustering of requirements glossary terms. IEEE Trans Software Eng 43(10):918–945
Barhom S, Shwartz V, Eirew A, Bugert M, Reimers N, Dagan I (2019) Revisiting joint modeling of cross-document entity and event coreference resolution. In: A. Korhonen, D.R. Traum, L. Màrquez (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. 4179–4189. Association for Computational Linguistics
Baziotis C, Pelekis N, Doulkeridis C (2017) Datastories at semeval-2017 task 4: Deep lstm with attention for message-level and topic-based sentiment analysis. In: proceedings of the 11th international workshop on semantic evaluation (SemEval-2017), pp. 747–754. Association for Computational Linguistics, Vancouver, Canada
Beheshti S, Benatallah B, Venugopal S, Ryu SH, Motahari-Nezhad HR, Wang W (2017) A systematic review and comparative analysis of cross-document coreference resolution methods and tools. Computing 99(4):313–349
Belsis P, Koutoumanos A, Sgouropoulou C (2014) PBURC: a patterns-based, unsupervised requirements clustering framework for distributed agile software development. Requir Eng 19(2):213–225
Berry DM, Kamsties E (2005) The syntactically dangerous all and plural in specifications. IEEE Software 22(1):55–57
Chantree F, Nuseibeh B, De Roeck A, Willis A(2006) Identifying nocuous ambiguities in natural language requirements. In: 14th IEEE international requirements engineering conference (RE’06)
Chen X, Liu Z, Sun M (2014) A unified model for word sense representation and disambiguation. In: A. Moschitti, B. Pang, W. Daelemans (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1025–1035. ACL
Cleland-Huang J Mining domain knowledge [requirements]. IEEE Software 32(3): 16–19
Cohen WW, Ravikumar P, Fienberg SE (2003)A comparison of string distance metrics for name-matching tasks. In: S. Kambhampati, C.A. Knoblock (eds.) Proceedings of IJCAI-03 Workshop on Information Integration on the Web (IIWeb-03), August 9-10, 2003, Acapulco, Mexico, pp. 73–78
Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa PP (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537
Deerwester SC, Dumais ST, Landauer TK, Furnas GW, Harshman RA (1990) Indexing by latent semantic analysis. JASIS 41(6):391–407
Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 , pp. 4171–4186 (Long and Short Papers)
Doddington GR, Mitchell A, Przybocki MA, Ramshaw LA, Strassel SM, Weischedel RM (2004) The automatic content extraction (ACE) program - tasks, data, and evaluation. In: proceedings of the fourth international conference on language resources and evaluation, LREC 2004, May 26-28, 2004, Lisbon, Portugal. European Language Resources Association
Ester M, Kriegel H, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: proceedings of the second international conference on knowledge discovery and data mining (KDD-96), Portland, Oregon, USA, pp. 226–231. AAAI Press
Fabbrini F, Fusani M, Gnesi S, Lami G (2001) The linguistic approach to the natural language requirements quality: Benefits of the use of an automatic tool. In: proceedings 26th annual NASA Goddard software engineering workshop, 2001
Falessi D, Cantone G, Canfora G (2013) Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Trans Software Eng 39(1):18–44
Femmer H, Fernández DM, Wagner S, Eder S (2017) Rapid quality assurance with requirements smells. J Syst Softw 123:190–213
Femmer H, Kucera J, Vetro A (2014) On the impact of passive voice requirements on domain modelling. In: M. Morisio, T. Dybå, M. Torchiano (eds.) 2014 ACM-IEEE international symposium on empirical software engineering and measurement, ESEM ’14, Torino, Italy, September 18-19, 2014, pp. 21:1–21:4. ACM
...Fernández DM, Wagner S, Kalinowski M, Felderer M, Mafra P, Vetro A, Conte T, Christiansson M, Greer D, Lassenius C, Männistö T, Nayabi M, Oivo M, Penzenstadler B, Pfahl D, Prikladnicki R, Ruhe G, Schekelmann A, Sen S, Spínola RO, Tuzcu A, de la Vara JL, Wieringa RJ (2017) Naming the pain in requirements engineering - contemporary problems, causes, and effects in practice. Empirical Softw Eng 22(5):2298–2338
Ferrari A, Esuli A (2019) An NLP approach for cross-domain ambiguity detection in requirements engineering. Autom Softw Eng 26(3):559–598
Gemkow T, Conzelmann M, Hartig K, Vogelsang A (2018) Automatic glossary term extraction from large-scale requirements specifications. In: 26th IEEE international requirements engineering conference, RE 2018, Banff, AB, Canada, August 20-24, 2018, pp. 412–417
Gleich B, Creighton O, Kof L (2010) Ambiguity detection: Towards a tool explaining ambiguity sources. In: R.J. Wieringa, A. Persson (eds.) Requirements Engineering: Foundation for Software Quality, 16th International Working Conference, REFSQ 2010, Essen, Germany, June 30 - July 2, 2010. Proceedings, Lecture Notes in Computer Science, vol. 6182, pp. 218–232. Springer
Gomaa W, Fahmy AA (2013) A survey of text similarity approaches. Int J Comput Appl 68(13)
Harris ZS (1954) Distributional structure. WORD 10(2–3):146–162
Hey T, Keim J, Koziolek A, Tichy WF (2020) Norbert: Transfer learning for requirements classification. In: 28th IEEE International Requirements Engineering Conference, RE 2020, Zurich, Switzerland, August 31 - September 4, 2020, pp. 169–179. IEEE
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Huang EH, Socher R, Manning CD, Ng AY(2012) Improving word representations via global context and multiple word prototypes. In: The 50th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, July 8-14, 2012, Jeju Island, Korea - Volume 1: Long Papers, pp. 873–882 (2012)
Huang Y, Chen C, Xing Z, Lin T, Liu Y (2018) Tell them apart: distilling technology differences from crowd-scale comparison discussions. In: M. Huchard, C. Kästner, G. Fraser (eds.) Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, Montpellier, France, September 3-7, 2018, pp. 214–224. ACM
Hull MEC, Jackson K, Dick J (2002) Requirements engineering. Springer, London
Johann T, Stanik C, B, AMA, Maalej W (2017)SAFE: A simple approach for feature extraction from app descriptions and app reviews. In: 25th IEEE International Requirements Engineering Conference, RE 2017, Lisbon, Portugal, September 4-8, 2017, pp. 21–30
Joshi M, Levy O, Zettlemoyer L, Weld DS (2019) BERT for coreference resolution: Baselines and analysis. In: K. Inui, J. Jiang, V. Ng, X. Wan (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp. 5802–5807. Association for Computational Linguistics
Jürgens E, Deissenboeck F, Feilkas M, Hummel B, Schätz B, Wagner S, Domann C, Streit J (2010) Can clone detection support quality assessments of requirements specifications? In: J. Kramer, J. Bishop, P.T. Devanbu, S. Uchitel (eds.) Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2, ICSE 2010, Cape Town, South Africa, 1-8 May 2010, pp. 79–88. ACM
Khlif W, Haoues M, Sellami A, Ben-Abdallah H (2017) Analyzing functional changes in BPMN models using COSMIC. In: J.S. Cardoso, L.A. Maciaszek, M. van Sinderen, E. Cabello (eds.) Proceedings of the 12th International Conference on Software Technologies, ICSOFT 2017, Madrid, Spain, July 24-26, 2017, pp. 265–274. SciTePress
Khlif W, Sellami A, Haoues M, Ben-Abdallah, H (2018) Using COSMIC FSM method to analyze the impact of functional changes in business process models. In: E. Damiani, G. Spanoudakis, L.A. Maciaszek (eds.) Proceedings of the 13th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2018, Funchal, Madeira, Portugal, March 23-24, 2018, pp. 124–136. SciTePress
Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings
Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. , IJCAI 95, Montréal Québec, Canada 2:1137–1145 (August 20-25 1995)
Lawrence, HubertPhipps, Arabie: comparing partitions. J Classif (1985)
Le QV, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014. JMLR Workshop and Conference Proceedings 32:1188–1196
Lee H, Recasens M, Chang AX, Surdeanu M, Jurafsky D (2012) Joint entity and event coreference resolution across documents. In: J. Tsujii, J. Henderson, M. Pasca (eds.) Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, July 12-14, 2012, Jeju Island, Korea, pp. 489–500. ACL
Lee K, He L, Lewis M, Zettlemoyer L (2017) End-to-end neural coreference resolution. In: M. Palmer, R. Hwa, S. Riedel (eds.) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pp. 188–197. Association for Computational Linguistics
Li M, Yang Y, Shi L, Wang Q, Hu J, Peng X, Liao W, Pi G (2020) Automated extraction of requirement entities by leveraging LSTM-CRF and transfer learning. In: IEEE International Conference on Software Maintenance and Evolution, ICSME 2020, Adelaide, Australia, September 28 - October 2, 2020, pp. 208–219. IEEE
Li M, Zhang T, Chen Y, Smola AJ (2014) Efficient mini-batch training for stochastic optimization. In: S.A. Macskassy, C. Perlich, J. Leskovec, W. Wang, R. Ghani (eds.) The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA - August 24 - 27, 2014, pp. 661–670. ACM
Li S, Zhao Z,Hu R, Li, W, Liu T, Du X (2018) Analogical reasoning on chinese morphological and semantic relations. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers, pp. 138–143
Lian X, Rahimi M, Cleland-Huang J, Zhang L, Ferrai R, Smith M (2016) Mining requirements knowledge from collections of domain documents. In: 24th IEEE International Requirements Engineering Conference, RE 2016, Beijing, China, September 12-16, 2016, pp. 156–165. IEEE Computer Society
Logeswaran L, Chang M, Lee K, Toutanova K, Devlin J, Lee H (2019) Zero-shot entity linking by reading entity descriptions. In: A. Korhonen, D.R. Traum, L. Màrquez (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. 3449–3460. Association for Computational Linguistics
Lu J, Ng V (2017) Joint learning for event coreference resolution. In: R. Barzilay, M. Kan (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pp. 90–101. Association for Computational Linguistics
Mahmoud A, Niu N (2015) On the role of semantics in automated requirements tracing. Requir Eng 20(3):281–300
Maletic JI, Marcus A (2000) Using latent semantic analysis to identify similarities in source code to support program understanding. In: 12th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2000), 13-15 November 2000, Vancouver, BC, Canada, pp. 46–53
Mallows EBFL (1983) A method for comparing two hierarchical clusterings. J Am Stat Assoc 78(383):553–569
Manning CD, Raghavan P, Schütze H (2010)Introduction to information retrieval
Marcus A, Maletic JI (2001) Identification of high-level concept clones in source code. In: 16th IEEE International Conference on Automated Software Engineering (ASE 2001), 26-29 November 2001, Coronado Island, San Diego, CA, USA, pp. 107–114
Marcus A, Maletic JI (2003) Recovering documentation-to-source-code traceability links using latent semantic indexing. In: Proceedings of the 25th International Conference on Software Engineering, May 3-10, 2003, Portland, Oregon, USA, pp. 125–137
Melamud O, Goldberger J, Dagan I(2016) context2vec: Learning generic context embedding with bidirectional LSTM. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August 11-12, 2016, pp. 51–61
Melamud O, McClosky D, Patwardhan S, Bansal M (2016)The role of context types and dimensionality in learning word embeddings. In: K. Knight, A. Nenkova, O. Rambow (eds.) NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pp. 1030–1040. The Association for Computational Linguistics
Mezghani M, Kang J, Sèdes F(2018) Industrial requirements classification for redundancy and inconsistency detection in SEMIOS. In: G. Ruhe, W. Maalej, D. Amyot (eds.) 26th IEEE International Requirements Engineering Conference, RE 2018, Banff, AB, Canada, August 20-24, 2018, pp. 297–303. IEEE Computer Society
Mihalcea R, Tarau P(2004) Textrank: Bringing order into text. Emnlp pp. 404–411
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. In: Y. Bengio, Y. LeCun (eds.) 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pp. 3111–3119
Misra J, Das S (2013) Entity disambiguation in natural language text requirements. In: P. Muenchaisri, G. Rothermel (eds.) 20th Asia-Pacific Software Engineering Conference, APSEC 2013, Ratchathewi, Bangkok, Thailand, December 2-5, 2013 - Volume 1, pp. 239–246. IEEE Computer Society
Nguyen XV, Epps J, Bailey J (2010) Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J Mach Learn Res 11:2837–2854
Palangi H, Deng L, Shen Y, Gao J, He X, Chen J, Song X, Ward RK(2015) Deep sentence embedding using the long short term memory network: Analysis and application to information retrieval. CoRR arXiv: abs/1502.06922
Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: A. Moschitti, B. Pang, W. Daelemans (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1532–1543. ACL
Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training
Rago A, Marcos CA, Diaz-Pace JA (2016) Identifying duplicate functionality in textual use cases by aligning semantic actions. Softw Syst Model 15(2):579–603
Rijsbergen C (1979) Information retrieval (2nd edition)
Rosadini B, Ferrari A, Gori G, Fantechi A, Gnesi S, Trotta I, Bacherini S (2017) Using NLP to detect requirements defects: An industrial experience in the railway domain. In: P. Grünbacher, A. Perini (eds.) Requirements Engineering: Foundation for Software Quality - 23rd International Working Conference, REFSQ 2017, Essen, Germany, February 27 - March 2, 2017, Proceedings,. Lecture Notes in Computer Science 10153:344–360 (Springer)
Rosenberg A, Hirschberg J (2007) V-measure: A conditional entropy-based external cluster evaluation measure. In: J. Eisner (ed.) EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Prague, Czech Republic, pp. 410–420. ACL
Sato-Ilic M (2000) On evaluation of clustering using homogeneity analysis. In: Proceedings of the IEEE International Conference on Systems, Man & Cybernetics: “Cybernetics Evolving to Systems, Humans, Organizations, and their Complex Interactions”, Sheraton Music City Hotel, Nashville, Tennessee, USA, 8-11 October 2000, pp. 3588–3593. IEEE
Shi L, Li M, Xing M, Wang Y, Wang Q, Peng X, Liao W, Pi G, Wang H (2020) Learning to extract transaction function from requirements: an industrial case on financial software. In: ESEC/FSE ’20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Virtual Event, USA, November 8-13, 2020, pp. 1444–1454. ACM
Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Tjong SF, Berry D M (2013) The design of SREE - A prototype potential ambiguity finder for requirements specifications and lessons learned. In: J. Dörr, A.L. Opdahl (eds.) Requirements Engineering: Foundation for Software Quality - 19th International Working Conference, REFSQ 2013, Essen, Germany, April 8-11, 2013. Proceedings. Lecture Notes in Computer Science 7830:80–95 (Springer)
Turian JP, Ratinov L, Bengio Y (2010) Word representations: A simple and general method for semi-supervised learning. In: J. Hajic, S. Carberry, S. Clark (eds.) ACL 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11-16, 2010, Uppsala, Sweden, pp. 384–394. The Association for Computer Linguistics
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Guyon I, von Luxburg U, Bengio S, Wallach HM, Fergus R, Vishwanathan SVN, Garnett R (eds) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017. Long Beach, CA, USA, pp 5998–6008
Wang W, Niu N, Liu H, Niu Z (2018) Enhancing automated requirements traceability by resolving polysemy. In: 26th IEEE International Requirements Engineering Conference, RE 2018, Banff, AB, Canada, August 20-24, 2018, pp. 40–51
Wang Y, Shi L, Li M, Wang Q, Yang Y (2020) A deep context-wise method for coreference detection in natural language requirements. In: 28th IEEE International Requirements Engineering Conference, RE 2020, Zurich, Switzerland, August 31 - September 4, 2020, pp. 180–191. IEEE
Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Scao TL, Gugger S, Drame M, Lhoest Q, Rush AM (2020) Transformers: State-of-the-art natural language processing. In: proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38–45. Association for Computational Linguistics, Online
Yang H, De Roeck A, Gervasi V, Willis A, Nuseibeh B (2011) Analysing anaphoric ambiguity in natural language requirements. Requir Eng 16(3):163–189
Acknowledgements
This work is supported by the National Key Research and Development Program of China under grant No. 2018YFB1403400, the National Science Foundation of China under grant No. 61802374, No. 61432001, No. 61602450. This work is also supported by China Merchants Bank Intelligent Software Research and Development Effectiveness Research Project.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, Y., Shi, L., Li, M. et al. Detecting coreferent entities in natural language requirements. Requirements Eng 27, 351–373 (2022). https://doi.org/10.1007/s00766-022-00374-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00766-022-00374-8