Abstract
This paper presents results from an industrial study that applied input space partitioning and semi-automated requirements modeling to large-scale industrial software, specifically financial calculation engines. Calculation engines are used in financial service applications such as banking, mortgage, insurance, and trading to compute complex, multi-conditional formulas to make high risk financial decisions. They form the heart of financial applications, and can cause severe economic harm if incorrect. Controllability and observability of these calculation engines are low, so robust and sophisticated test methods are needed to ensure the results are valid. However, the industry norm is to use pure human-based, requirements-driven test design, usually with very little automation. The Federal Home Loan Mortgage Corporation (FHLMC), commonly known as Freddie Mac, concerned that these test design techniques may lead to ineffective and inefficient testing, partnered with a university to use high quality, sophisticated test design on several ongoing projects. The goal was to determine if such test design can be cost-effective on this type of critical software. In this study, input space partitioning, along with automation, were applied with the help of several special-purpose tools to validate the effectiveness of input space partitioning. Results showed that these techniques were far more effective (finding more software faults) and more efficient (requiring fewer tests and less labor), and the managers reported that the testing cycle was reduced from five human days to 0.5. This study convinced upper management to begin infusing this approach into other software development projects.
Similar content being viewed by others
Notes
References
Alluri C (2008) Testing calculation engines using input space partitioning and automation. Master’s thesis, Department of Information and Software Engineering, George Mason University, Fairfax VA. Available on the web at: http://www.cs.gmu.edu/~offutt/
Ammann P, Offutt J (2008) Introduction to software testing. Cambridge University Press, Cambridge, UK. ISBN 0-52188-038-1
Ammann P, Offutt J, Huang H (2003) Coverage criteria for logical expressions. In: Proceedings of the 14th international symposium on software reliability engineering, Denver, CO, IEEE Computer Society Press, Los Alamitos, CA, pp 99–107
Bach J (2005) Allpairs test case generation tool. http://www.satisfice.com/tools.shtml. Accessed June 2012
Beizer B (1990) Software testing techniques, 2nd edn. Van Nostrand Reinhold, Inc, New York NY. ISBN 0-442-20672-0
Czerwoka J (2006) Pairwise testing in real world: practical extensions to test case generators. In: Proceedings of the 24th annual pacific Northwest software quality conference, Portland OR, USA, pp 419–430
Freedman RS (1991) Testability of software components. IEEE Trans Softw Eng 17(6):553–564
Grindal M, Offutt J, Andler SF (2005) Combination testing strategies: a survey. Softw Test Verif Reliab 15(2):97–133
Grindal M, Offutt J, Mellin J (2007) Conflict management when using combination strategies for software testing. In: Australian Software Engineering Conference (ASWEC 2007), Melbourne, Australia, pp 255–264
IBM (2011) Rational robot. Online http://www-01.ibm.com/software/awdtools/tester/robot/. Accessed July 2011
Kacker R, Kuhn R (2008) Automated combinatorial testing for software-beyond pairwise testing. Online http://csrc.nist.gov/groups/SNS/acts/. Accessed June 2009
Lander J, Orphanides A, Douvogiannis M (1997) Earnings, forecasts and the predictability of stock returns: evidence from trading the s&p. J Portf Manage 23:24–35
Myers G (1979) The art of software testing. Wiley, New York, NY
Ostrand TJ, Balcer MJ (1988) The category-partition method for specifying and generating functional tests. Commun ACM 31(6):676–686
Ostrand TJ, Sigal R, Weyuker EJ (1986) Design for a tool to manage specification-based testing. In: Proceedings of the workshop on software testing, Banff, Alberta. IEEE Computer Society Press, Los Alamitos, CA, pp 41–50
Sortino F, Price L (1994) Performance measurement in a downside risk framework. J Invest 3(3):59–64
Voas JM (1992) PIE: a dynamic failure-based technique. IEEE Trans Softw Eng 18(8):717–727
Author information
Authors and Affiliations
Corresponding author
Additional information
Editor:James Miller
Rights and permissions
About this article
Cite this article
Offutt, J., Alluri, C. An industrial study of applying input space partitioning to test financial calculation engines . Empir Software Eng 19, 558–581 (2014). https://doi.org/10.1007/s10664-012-9229-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-012-9229-5