Abstract
Agile methods are now widely used in software engineering organizations, whereas most formal methods are limited to niches and are perceived as inadequate in the context of agile development. This paper presents a case study of the innovative practices used at Anaplan, a financial planning and analysis software provider, to integrate formal specification within an agile process. The results show how Behavior-Driven Specification (BDS), by documenting behavior using executable acceptance criteria (EAC), is used to validate the design and implementation of calculation functions in Anaplan’s sparse calculation engine, while keeping all stakeholders aligned on the requirements. We also show that the interaction between the specifiers and the developers allows catching implementation issues at early stages of the development, while allowing the specification to remain amenable to emerging implementation constraints. The validated requirements have enabled the development of a framework to automatically generate extensive test coverage that is used to verify the implementation. As a result, over 200 bugs were caught in the production code before release, not counting the hundreds of issues that BDS allowed developers to detect earlier in the process. We show that BDS leads to high levels of confidence in the behavioral correctness of software while being fully aligned with agile practices, and proves to be a significant evolution in the field of software development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Approximately 50 engineers were involved over the lifetime of the project.
- 2.
Language fracture exists when stakeholders use different formalisms.
- 3.
Note that the keyword whenever is used in the scenario as when is a Kotlin keyword.
- 4.
We are unable to provide more accurate examples due to the proprietary nature of the work, but the simplicity of these scenarios is comparable to that of many of our actual EACs.
- 5.
The optimization of said combination being a common source of bugs.
- 6.
The inner workings of the verification framework are complex and will be the subject of an upcoming paper.
- 7.
Each development team has its own unit and functional test suites, so the regression suite catches issues that the existing testing mechanisms have failed to detect.
- 8.
To underline the significance of this number, fewer than 500 calculation bugs have been raised against the dense engine over the past 5 years.
- 9.
This only applies to changes to existing behavior such as bug fixes. New behavior can be implemented and specified in parallel.
References
Alkobaisi, S., Bae, W.D., Narayanappa, S., Debnath, N.: Steel threads: software engineering constructs for defining, designing and developing software system architecture. J. Comput. Methods Sci. Eng. 12(s1), S63–S77 (2012). https://doi.org/10.3233/JCM-2012-0437
Alwidian, S., Jaskolka, J.: Understanding the role of human-related factors in security requirements elicitation. In: International Working Conference on Requirements Engineering: Foundation for Software Quality, pp. 65–74. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-29786-1_5
Beck, K., et al.: Manifesto for agile software development (2001). http://www.agilemanifesto.org/
Buchan, J.: An empirical cognitive model of the development of shared understanding of requirements. In: Zowghi, D., Jin, Z. (eds.) Requirements Engineering. CCIS, vol. 432, pp. 165–179. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-43610-3_13
Codd, E.F., Codd, S.B., Salley, C.T.: Providing OLAP (on-line analytical processing) to user-analysts. An IT Mandate. White Paper. Arbor Software Corporation 4 (1993)
Evans, E.: Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley Professional (2004)
Fraser, S., Pezzoni, A.: Azuki framework to assist with behavior-driven specification. https://github.com/anaplan-engineering/azuki. Accessed 01 Oct 2023
Fraser, S., Pezzoni, A.: Behaviour driven specification. In: Proceedings of the 19th International Overture Workshop, pp. 5–20 (2021). https://doi.org/10.48550/arXiv.2110.09371
Ghezzi, C.: Formal methods and agile development: towards a happy marriage. In: The Essence of Software Engineering, pp. 25–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73897-0_2
Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576–580 (1969). https://doi.org/10.1145/363235.363259
Inayat, I., Salim, S.S., Marczak, S., Daneva, M., Shamshirband, S.: A systematic literature review on agile requirements engineering practices and challenges. Comput. Hum. Behav. 51, 915–929 (2015). https://doi.org/10.1016/j.chb.2014.10.046
Jebreen, I., Awad, M., Al-Qerem, A.: A propose model for shared understanding of software requirements (SUSRs). In: Information Science and Applications (ICISA) 2016, pp. 1045–1056. Springer, Singapore (2016). https://doi.org/10.1007/978-981-10-0557-2_100
Kotonya, G., Sommerville, I.: Requirements Engineering: Processes and Techniques. Wiley (1998)
Larsen, P.G., Battle, N., Ferreira, M., Fitzgerald, J., Lausdahl, K., Verhoef, M.: The overture initiative integrating tools for VDM. ACM SIGSOFT Softw. Eng. Notes 35(1), 1–6 (2010). https://doi.org/10.1145/1668862.1668864
Sidorova, M.I., et al.: Strategic planning software as a tool for improvement of business information space. In: European Proceedings of Social and Behavioural Sciences (2022)
Wang, X., Zhao, L., Wang, Y., Sun, J.: The role of requirements engineering practices in agile development: an empirical study. In: Zowghi, D., Jin, Z. (eds.) Requirements Engineering. CCIS, vol. 432, pp. 195–209. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-43610-3_15
Wynne, M., Hellesoy, A., Tooke, S.: The Cucumber Book: Behaviour-Driven Development for Testers and Developers. Pragmatic Bookshelf (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Allred, J.D., Fraser, S., Pezzoni, A. (2024). Behavior-Driven Specification in Practice: An Experience Report. In: Mendez, D., Moreira, A. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2024. Lecture Notes in Computer Science, vol 14588. Springer, Cham. https://doi.org/10.1007/978-3-031-57327-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-57327-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-57326-2
Online ISBN: 978-3-031-57327-9
eBook Packages: Computer ScienceComputer Science (R0)