Abstract
Methods and tools for detecting and measuring ambiguity in texts have been proposed for years, yet their efficacy is still under study for improvement, encouraged by results in various application fields (requirements, legal documents, interviews, ...). The paper presents a fresh-started process aimed at validating such methods and tools by applying some of them to a semi-structured data corpus. This corpus represents results of manual reviews, done by international experts, along with their source texts. The purpose is to check how much results of automated analysis are consistent with the reviewers reports. The application domain is that of safety-related system/software Standards in Railway. Thus, if we increase confidence in tools, then we also increase confidence in Standard correctness, which in turn impacts in conforming products. Care is taken in using, for scientific purpose only, sensitive, unpublished source data (the comments, protected by NDAs), that are kept reviewer-anonymous before statistical results are produced, while the Standards are publicly available texts. The results will also be used to improve the tools themselves, even if much elaboration is still to be carried out: the research is still at its beginning, so metrics for tool evaluation is a goal, whose characteristics are just sketched and discussed in the paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fantechi, A., Gnesi, S., Ristori, G., Carenini, M., Vanocchi, M., Moreschini, P.: Assisting requirement formalization by means of natural language translation. Formal Methods Syst. Des. 4(3), 243–263 (1994)
Fabbrini, F., Fusani, M., Gnesi, S., Lami, G.: An automatic quality evaluation for natural language requirements. In: Proceedings of 7th REFSQ (2001)
Fantechi, A., Ferrari, A., Gnesi, S., Semini, L.: Hacking an ambiguity detection tool to extract variation points: an experience report. In: Proceedings of the 12th International Workshop on Variability Modelling of Software-Intensive Systems, pp. 1–13 (2018)
Ferrari, A., Trentanni, G., Gnesi, S.: An automatic quality evaluation for natural language requirements. In: Proceedings of 1st Workshop on Natural Language Processing for Requirements Engineering and NLP Tool Showcase, RESFQ 2018, March 19th - Utrecht, The Netherlands (2019)
Gnesi, S., Lami, G., Trentanni, G.: An automatic tool for the analysis of natural language requirements. IJCSSE 20(1) (2005)
CENELEC: EN 50128 - Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems (2011)
CENELEC: EN 50126–1 - Railway Applications - The Specification and Demonstration of Reliability, Availability, Maintainability and Safety (RAMS) - Part 1: Generic RAMS Process (2017)
CENELEC: Internal Regulations Part 2: Common Rules For Standardization Works (2017)
CENELEC: Internal Regulations Part 3: Principles and rules for the structure and drafting of CEN and CENELEC documents (2017)
Fenton, N., Neil, M.: A strategy for improving safety related software engineering standards. IEEE Trans. Software Eng. 24(11), 1002–1013 (1998)
Fantechi, A., Gnesi, S., Lami, G., Maccari, A.: Application of linguistic techniques for use case analysis. In: Proceedings of IEEE 10th RE, pp. 157–164 (2002)
Ferrari, A., Trentanni, G., Gnesi, S.: Research on NLP for RE at CNR-ISTI: a Report. In: Proceedings of 1st Workshop on Natural Language Processing for Requirements Engineering and NLP Tool Showcase, RESFQ 2018, 19th March 2018, Utrecht, The Netherlands (2018)
Gnesi, S: Trentanni, G.: QuARS: a NLP tool for requirements analysis. In: Proceedings of 2nd Workshop on Natural Language Processing for Requirements Engineering and NLP Tool Showcase, RESFQ 2019, 18th March 2019, Essen, Germany (2019)
Graydon, P., Holloway, C.: Planning the unplanned experiment: assessing the efficacy of standards for safety critical software. NASA/TM-2015-218804, September 2015
Biscoglio, I., Coco, A., Fusani, M., Gnesi, S., Trentanni, G.: An approach to ambiguity analysis in safety-related standards. In: Proceedings of International Conference on the Quality of Information and Communications Technology (QUATIC 2010), pp. 146–176 (2010)
Ferrari, A., Fusani, M., Gnesi, S.: Are standards an ambiguity-free reference for product validation? In: Fantechi, A., Lecomte, T., Romanovsky, A. (eds.) RSSRail. Lecture Notes in Computer Science, vol. 10598. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68499-4_17
Ferrari, A., et al.: Detecting requirements defects with NLP patterns: an industrial experience in the railway domain. IEEE Empir. Softw. Eng. 23(6), 3684–3733 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Biscoglio, I., Ciancabilla, A., Fusani, M., Lami, G., Trentanni, G. (2019). Comparing Results of Natural Language Disambiguation Tools with Reports of Manual Reviews of Safety-Related Standards. In: ter Beek, M., Fantechi, A., Semini, L. (eds) From Software Engineering to Formal Methods and Tools, and Back. Lecture Notes in Computer Science(), vol 11865. Springer, Cham. https://doi.org/10.1007/978-3-030-30985-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-30985-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30984-8
Online ISBN: 978-3-030-30985-5
eBook Packages: Computer ScienceComputer Science (R0)