Skip to main content
SpringerLink
Log in

Measuring and improving software requirements elicitation in a small-sized software organization: a lightweight implementation of ISO/IEC/IEEE 15939:2017—systems and software engineering—measurement process

  • Original Article
  • Published:
Requirements Engineering Aims and scope Submit manuscript

Abstract

Requirements engineering is one of the most important areas of software engineering because it enables practitioners to determine the characteristics and constraints of the software to be developed. Therefore, the requirements elicitation process aims to obtain a preliminary version of the requirements before their specification, but it is difficult to know if the elicited requirements have the desired attributes (i.e., these requirements are complete, correct, consistent, etc.) due to the nature of this process. There are some measures that allow requirements engineers to quantify certain aspects of the requirements elicitation process and its final product (i.e., the preliminary version of the requirements, also known as preliminary requirements), but there is no way of measuring and controlling the quality of the elicited preliminary requirements. With the aim of understanding, evaluating, and improving the activities of the requirements elicitation process, some of the existing measures for this process were analyzed and, consequently, a case study was conducted to define and implement a measurement program with a set of eight measures that are proposed in this study. This program is based on the ISO/IEC/IEEE 15939:2017—systems and software engineering—measurement process, an international standard that establishes a common process and framework for the measurement of systems and software. The measurement program corresponds to a lightweight implementation of the standard in the context of a small-sized software organization (where the size of the software staff is 11–50 people, the size of projects is 50,000–100,000 LOC, and the time spent on the projects is six to twelve months) and the obtained results showed a preliminary positive influence when obtaining high-quality preliminary requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability statement

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Notes

  1. This activity corresponds to the activity “1. Analyze the complete set of elicited requirements” defined by section “6.2.3.3 Analyze and maintain stakeholder requirements” from the ISO/IEC/IEEE 29148:2018 standard.

  2. The requirements engineers fulfilled the roles of elicitors and analysts of the preliminary requirements, and they have skills and responsibilities to perform the requirements elicitation as well as writing skills for specifying the requirements.

  3. The Volere template for stakeholder analysis is a spreadsheet that enables the requirements engineer to match types of knowledge with the roles in organization that provide the needed knowledge. According to Robertson and Robertson [53], the analysis spreadsheet is an effective way of recording and keeping track of all the stakeholders.

References

  1. Leyva M, Hadar I, Aviv I (2019) A requirements engineering methodology for knowledge management solutions: Integrating technical and social aspects. Requirements Eng 24(4):503–521. https://doi.org/10.1007/s00766-018-0298-x

    Article  Google Scholar 

  2. Kotonya G, Sommerville I (1998) Requirements engineering: processes and techniques. John Wiley and Sons, Chichester

    Google Scholar 

  3. Berenbach B, Paulish DJ, Kazmeier J, Rudorfer A (2009) Software & systems requirements engineering: in practice. McGraw-Hill Inc, New York, NY

    Google Scholar 

  4. Ali N, Cai H, Hamou-Lhadj A, Hassine J (2019) Exploiting parts-of-speech for effective automated requirements traceability. Inf Softw Technol 106:126–141. https://doi.org/10.1016/j.infsof.2018.09.009

    Article  Google Scholar 

  5. Femmer H, Hauptmann B, Eder S, Moser D (2016) Quality assurance of requirements artifacts in practice: a case study and a process proposal. In: Abrahamsson P, Jedlitschka A, Duc AN, Felderer M, Amasaki S, Mikkonen T (eds) Product-Focused Software Process Improvement. Springer, Cham, pp 506–516. https://doi.org/10.1007/978-3-319-49094-6_36

    Chapter  Google Scholar 

  6. Medeiros J, Vasconcelos A, Silva C, Goulão M (2018) Quality of software requirements specification in agile projects: a cross-case analysis of six companies. J Syst Softw 142:171–194. https://doi.org/10.1016/j.jss.2018.04.064

    Article  Google Scholar 

  7. Bourque P, Fairle RE (2014) Guide to the software engineering body of knowledge (SWEBOK®). IEEE Computer Society Press, Los Alamitos

    Google Scholar 

  8. Sandhu RK, Weistroffer HR (2018) A review of fundamental tasks in requirements elicitation. In: Wrycza S, Maślankowski J (eds) Information Systems: Research, Development, Applications, Education. Springer, Cham, pp 31–44

    Chapter  Google Scholar 

  9. Dar H, Lali MI, Ashraf H, Ramzan M, Amjad T, Shahzad B (2018) A systematic study on software requirements elicitation techniques and its challenges in mobile application development. IEEE Access 6:63859–63867. https://doi.org/10.1109/ACCESS.2018.2874981

    Article  Google Scholar 

  10. International Organization for Standardization (2018). ISO/IEC/IEEE 29148:2018 - Systems and software engineering - Life cycle processes -Requirements engineering, Geneva, Switzerland

  11. Chen F, Power N, Collins J J, Ishikawa F (2019) Contemporary requirements challenges and issues: an empirical study in 11 organizations. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (SAC ’19). ACM, pp 1592–1599. https://doi.org/10.1145/3297280.3297435

  12. Ahmad S, Asmai SA (2016) Measuring software requirements quality following negotiation through empirical study. Int J Appl Eng Res 11(6):4190–4196

    Google Scholar 

  13. Palmer J D, Evans R P. (1994). Software risk management: requirements-based risk metrics, In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Humans, Information and Technology. IEEE, pp 836–841. https://doi.org/10.1109/ICSMC.1994.399939

  14. Dutoit AH, Bruegge B (1998) Communication metrics for software development. IEEE Trans Software Eng 24(8):615–628. https://doi.org/10.1109/32.707697

    Article  Google Scholar 

  15. Kaiya H, Saeki M (2006) Using domain ontology as domain knowledge for requirements elicitation. In: 14th IEEE International Requirements Engineering Conference. IEEE, pp. 189–198. https://doi.org/10.1109/RE.2006.72

  16. Barragáns-Martínez AB, Pazos-Arias JJ, Fernández-Vilas A, García-Duque J, López-Nores M, Díaz-Redondo RP, Blanco-Fernández Y (2008) On the interplay between inconsistency and incompleteness in multi-perspective requirements specifications. Inf Softw Technol 50(4):296–321. https://doi.org/10.1016/j.infsof.2007.02.001

    Article  Google Scholar 

  17. Nakatani T, Tsumaki T, Tsuda M, Inoki M, Hori S, Katamine K (2011) Requirements maturation analysis by accessibility and stability. In: 18th IEEE Asia-Pacific Software Engineering Conference. IEEE, pp 357–364. https://doi.org/10.1109/APSEC.2011.56

  18. Zapata S, Torres E, Sevilla G, Aballay L, Reus M (2012) Effectiveness of traditional software requirement elicitation techniques applied in distributed software development scenarios, In: Proceedings of the XXXVIII Conferencia Latinoamericana en Informatica (CLEI). IEEE, pp 1–7. https://doi.org/10.1109/CLEI.2012.6427200

  19. Obana M, Hanakawa N (2014) Process evaluation based on meeting quality of requirement analysis phase in software development projects. J Softw Eng Appl 7(10):828–843. https://doi.org/10.4236/jsea.2014.710075

    Article  Google Scholar 

  20. Byun J, Rhew S, Hwang M, Sugumara V, Park S, Park S (2014) Metrics for measuring the consistencies of requirements with objectives and constraints. Requirements Eng 19(1):89–104. https://doi.org/10.1007/s00766-013-0180-9

    Article  Google Scholar 

  21. Lauesen S (2002) Software requirements, styles and techniques. Pearson Education, Harlow

    Google Scholar 

  22. Institute of Electrical and Electronics Engineers (1998). IEEE Std. 830–1998 - Guide for developing system requirements specifications, Piscataway, NJ

  23. Gotel, O. & Finkelstein, C. W. (1994). An analysis of the requirements traceability problem, In: proceedings of the first international conference on requirements engineering. IEEE, pp 94–101. https://doi.org/10.1109/ICRE.1994.292398

  24. Kneuper R (2018) Software and software process measurement software processes and life cycle models. Springer, Berlin, pp 261–285

    Book  Google Scholar 

  25. Tahir T, Rasool G, Gencel C (2016) A systematic literature review on software measurement programs. Inf Softw Technol 73:101–121. https://doi.org/10.1016/j.infsof.2016.01.014

    Article  Google Scholar 

  26. Nordin A, Zaidi NHA, Mazlan NA (2017) Measuring software requirements specification. J Telecommun Electron Comput Eng 9(3–5):123–128

    Google Scholar 

  27. International Organization for Standardization (2001). ISO/IEC 14598 - Information Technology - Software Product Evaluation, Geneva, Switzerland

  28. International Organization for Standardization (2004). ISO/IEC 9126 - Software Engineering - Product Quality, Geneva, Switzerland

  29. International Organization for Standardization Standard (2007). ISO/IEC 15939:2007 - Systems and Software Engineering - Measurement Process, Geneva, Switzerland

  30. CMMI Product Team (2010). Capability Maturity Model Integration (CMMI) for Development, Version 1.3, Technical Report CMU/SEI-2010-TR-033, Software Engineering Institute, Carnegie Mellon University

  31. International Organization for Standardization (2014). ISO/IEC 25021:2014 - Systems and Software Engineering. Requirements and Evaluation of Systems Quality and Software Security (SQuaRE) - Quality Score Elements, Geneva, Switzerland

  32. Palyagar B (2004) Measuring and influencing requirements engineering process quality in organizations. In: 9th Australian Workshop on Requirements Engineering (AWRE ’04), Adelaide, Australia, pp. 95–109.

  33. Niazi M (2005) An instrument for measuring the maturity of requirements engineering process. In: Bomarius F, Komi-Sirviö S (eds) Product-Focused Software Process Improvement. Springer, Berlin, pp 574–585

    Chapter  Google Scholar 

  34. Niazi M, Cox K, Verner J (2008) A measurement framework for assessing the maturity of requirements engineering process. Softw Qual J 16:213–235. https://doi.org/10.1007/s11219-007-9033-4

    Article  Google Scholar 

  35. Génova G, Fuentes JM, Llorens J, Hurtado O, Moreno V (2013) A framework to measure and improve the quality of textual requirements. Requirements Eng 18(1):25–41. https://doi.org/10.1007/s00766-011-0134-z

    Article  Google Scholar 

  36. Bhatti SN, Usman M, Jadi AA (2015) Validation to the requirements elicitation framework via metrics. ACM SIGSOFT Softw Eng Notes 40(5):1–7. https://doi.org/10.1145/2815021.2815031

    Article  Google Scholar 

  37. Pinto N, Tortosa N, Geat B C, Ibáñez L, Bollati V (2018) Quality evaluation of agile processes: Measurement of requirements management using AQF v2. In: 11th International Conference on the Quality of Information and Communications Technology (QUATIC). IEEE, pp 15–20. https://doi.org/10.1109/QUATIC.2018.00013

  38. Tahir T, Rasool G, Noman M (2018) A systematic mapping study on software measurement programs in SMEs. e-Informatica Softw Eng J 12(1):133–165. https://doi.org/10.5277/e-Inf180106

    Article  Google Scholar 

  39. Laporte CY, Alexandre S, Renault A (2008) Developing international standards for very small enterprises. Computer 41(3):98–101. https://doi.org/10.1109/MC.2008.86

    Article  Google Scholar 

  40. Laporte C Y, Renault A, Alexandre S (2008b) The Application of International Software Engineering Standards in Very Small Enterprises. Software Process Improvement for Small and Medium Enterprises: Techniques and Case Studies (42–70). IGI Global

  41. International Organization for Standardization Standard (2017) ISO/IEC/IEEE 15939:2017 - Systems and software engineering - measurement process, Geneva, Switzerland

  42. Enríquez JG, Sánchez-Begines JM, Domínguez-Mayo FJ, García-García JA, Escalona MJ (2019) An approach to characterize and evaluate the quality of product lifecycle management software systems. Comput Stand Interfaces 61:77–88. https://doi.org/10.1016/j.csi.2018.05.003

    Article  Google Scholar 

  43. Chorás M, Springer T, Kozik R, López L, Martínez-Fernández S, Ram P, Rodríguez P, Franch X (2020) Measuring and improving agile processes in a small-size software development company. IEEE Access 8:78452–78466. https://doi.org/10.1109/ACCESS.2020.2990117

    Article  Google Scholar 

  44. Wohlin C, Höst M, Henningsson K (2003) Empirical research methods in software engineering. In: Conradi R, Wang AI (eds) Empirical Methods and Studies in Software Engineering. Springer, Berlin, pp 7–23

    Chapter  Google Scholar 

  45. Díaz M, García F, Piattini M (2005) Defining, performing and maintaining software measurement programs: State of art, In: VI Simposio Internacional de Sistemas de Información e Ingeniería de Software en la Sociedad del Conocimiento, Cartagena de Indias, Colombia, pp 13–23

  46. Basili VR (1993) Applying the goal/question/metric paradigm in the experience factory, software quality assurance and measurement: a worldwide perspective. International Thomson Publishing, London

    Google Scholar 

  47. Basili VR, Weiss DM (1984) A methodology for collecting valid software engineering data. IEEE Trans Software Eng 10(6):728–738. https://doi.org/10.1109/TSE.1984.5010301

    Article  Google Scholar 

  48. Basili V R, Heidrich J, Lindvall M, Munch J, Regardie M, Trendowicz A (2007) GQM^+ strategies -- aligning business strategies with software measurement, In: First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007). IEEE, pp 488–490. https://doi.org/10.1109/ESEM.2007.66

  49. Basili VR, Caldiera VRBG, Rombach HD (1994) Goal question metric paradigm. Encycl Softw Eng 1:528–532

    Google Scholar 

  50. Méndez Fernández D, Wagner S, Kalinowski M, Felderer M, Mafra P, Vetrò A, Conte T, Christiansson MT, Greer D, Lassenius C, Männistö T, Nayabi M, Oivo M, Penzenstadler B, Pfahl D, Prikladnicki R, Ruhe G, Schekelmann A, Sen S, Spinola R, Tuzcu A, de la Vara JL, Wieringa R (2017) Naming the pain in requirements engineering. Empir Softw Eng 22(5):2298–2338. https://doi.org/10.1007/s10664-016-9451-7

    Article  Google Scholar 

  51. Pacheco C, Garcia I, Reyes M (2018) Requirements elicitation techniques: a systematic literature review based on the maturity of the techniques. IET Software 12(4):365–378. https://doi.org/10.1049/iet-sen.2017.0144

    Article  Google Scholar 

  52. Wagner S, Méndez Fernández D, Felderer M, Vetrò A, Kalinowski M, Wieringa R, Pfahl D, Conte T, Christiansson MT, Greer D, Lassenius C, Männistö T, Nayebi M, Oivo M, Penzenstadler B, Prikladnicki R, Ruhe G, Schekelmann A, Sen S, Spínola R, Tuzcu A, Vara JL, Winkler D (2019) Status quo in requirements engineering: A theory and a global family of surveys. ACM Trans Softw Eng Methodol 28(2):1–48. https://doi.org/10.1145/3306607

    Article  Google Scholar 

  53. Robertson S, Robertson J (2006) Mastering the requirements process, 2nd edn. Addison-Wesley, New York

    Google Scholar 

  54. Meding W, Staron M. (2020) Making software measurement standards understandable. In: Proceedings of the international conference on evaluation and assessment in software engineering (EASE ’20). ACM, pp 368–370. https://doi.org/10.1145/3383219.3383269

  55. McGarry J, Card D, Jones C, Layman B, Clark E, Dean J, Hall F (2002) Practical software measurement: objective information for decision makers. Addison-Wesley, Boston

    Google Scholar 

  56. Pacheco C, Garcia I (2012) A systematic literature review of stakeholder identification methods in requirements elicitation. J Syst Softw 85(9):2171–2181. https://doi.org/10.1016/j.jss.2012.04.075

    Article  Google Scholar 

Download references

Funding

None.

Author information

Authors and Affiliations

  1. División de Estudios de Posgrado, Universidad Tecnológica de La Mixteca, Huajuapan de León, Mexico

    C. Pacheco, I. Garcia & M. Reyes

  2. Escuela Técnica Superior de Ingenieros Informáticos, Universidad Politécnica de Madrid, Madrid, Spain

    J. A. Calvo-Manzano

Authors
  1. C. Pacheco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. A. Calvo-Manzano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Reyes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to I. Garcia.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pacheco, C., Garcia, I., Calvo-Manzano, J.A. et al. Measuring and improving software requirements elicitation in a small-sized software organization: a lightweight implementation of ISO/IEC/IEEE 15939:2017—systems and software engineering—measurement process. Requirements Eng 28, 257–281 (2023). https://doi.org/10.1007/s00766-022-00394-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-022-00394-4

Keywords

Search

Navigation