Skip to main content
Springer Nature Link
Log in

A Literature Survey of Assertions in Software Testing

  • Conference paper
  • First Online:
Engineering of Computer-Based Systems (ECBS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14390))

Included in the following conference series:

Abstract

Assertions are one of the most useful automated techniques for checking program’s behaviour and hence have been used for different verification and validation tasks. We provide an overview of the last two decades of research involving ‘assertions’ in software testing. Based on a term–based search, we filtered the inclusion of relevant papers and synthesised them w.r.t. the problem addressed, the solution designed, and the evaluation conducted. The survey rendered 119 papers on assertions in software testing. After test oracle, the dominant problem focus is test generation, followed by engineering aspects of assertions. Solutions are typically embedded in tool prototypes and evaluated throughout limited number of cases while using large–scale industrial settings is still a noticeable method. We conclude that assertions would be worth more attention in future research, particularly regarding the new and emerging demands (e.g., verification of programs with uncertainty), for effective, applicable, and domain-specific solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Common Weakness Enumerations – https://cwe.mitre.org.

  2. 2.

    https://www.evosuite.org/experimental-data/sf110/.

References

  1. Abdi, M., Rocha, H., Demeyer, S., Bergel, A.: Small-Amp: test amplification in a dynamically typed language. Empir. Softw. Eng. 27(6), 128 (2022). https://doi.org/10.1007/s10664-022-10169-8

    Article  Google Scholar 

  2. Alagarsamy, S., Tantithamthavorn, C., Aleti, A.: A3Test: assertion-augmented automated test case generation (2023). https://doi.org/10.48550/ARXIV.2302.10352

  3. Almasi, M.M., Hemmati, H., Fraser, G., Arcuri, A., Benefelds, J.: An industrial evaluation of unit test generation: finding real faults in a financial application. In: IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice Track, pp. 263–272 (2017). https://doi.org/10.1109/ICSE-SEIP.2017.27

  4. Anand, S., et al.: An orchestrated survey of methodologies for automated software test case generation. J. Syst. Softw. 86(8), 1978–2001 (2013). https://doi.org/10.1016/j.jss.2013.02.061

    Article  Google Scholar 

  5. Araujo, W., Briand, L., Labiche, Y.: On the effectiveness of contracts as test oracles in the detection and diagnosis of race conditions and deadlocks in concurrent object-oriented software. In: International Symposium on Empirical Software Engineering and Measurement, pp. 10–19 (2011). https://doi.org/10.1109/ESEM.2011.9

  6. Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 41(5), 507–525 (2015). https://doi.org/10.1109/TSE.2014.2372785

    Article  Google Scholar 

  7. Bavota, G., Qusef, A., Oliveto, R., De Lucia, A., Binkley, D.: Are test smells really harmful? an empirical study. Empir. Softw. Eng. 20(4), 1052–1094 (2015). https://doi.org/10.1007/s10664-014-9313-0

    Article  Google Scholar 

  8. Blasi, A., Gorla, A., Ernst, M.D., Pezzè, M., Carzaniga, A.: MeMo: automatically identifying metamorphic relations in javadoc comments for test automation. J. Sys. Softw. 181 (2021). https://doi.org/10.1016/j.jss.2021.111041

  9. Boshernitsan, M., Doong, R., Savoia, A.: From daikon to agitator: lessons and challenges in building a commercial tool for developer testing. In: ACM International Symposium on Software Testing and Analysis, pp. 169–180. ISSTA ’06, ACM (2006). https://doi.org/10.1145/1146238.1146258

  10. Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on java predicates. ACM SIGSOFT Softw. Eng. Notes 27(4), 123–133 (2002). https://doi.org/10.1145/566171.566191

    Article  Google Scholar 

  11. Catolino, G., Palomba, F., Zaidman, A., Ferrucci, F.: How the experience of development teams relates to assertion density of test classes. In: IEEE International Conference on Software Maintenance and Evolution, pp. 223–234 (2019). https://doi.org/10.1109/ICSME.2019.00034, ISSN: 2576-3148

  12. Chen, J., Bai, Y., Hao, D., Zhang, L., Zhang, L., Xie, B.: How do assertions impact coverage-based test-suite reduction? In: IEEE International Conference on Software Testing, Verification and Validation, pp. 418–423 (2017). https://doi.org/10.1109/ICST.2017.45

  13. Cheon, Y., Kim, M., Perumandla, A.: A complete automation of unit testing for java programs. Tech. Rep. UTEP-CS-05-05, University of Texas at El Paso (2005). https://scholarworks.utep.edu/cs_techrep/234

  14. Cheon, Y., Leavens, G.T.: A simple and practical approach to unit testing: the JML and JUnit way. In: Magnusson, B. (ed.) ECOOP 2002. LNCS, vol. 2374, pp. 231–255. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47993-7_10

    Chapter  Google Scholar 

  15. Choudhary, S.R., Zhao, D., Versee, H., Orso, A.: WATER: web application TEst repair. In: ACM International Workshop on End-to-End Test Script Engineering, pp. 24–29. ETSE ’11, ACM (2011). https://doi.org/10.1145/2002931.2002935

  16. Clarke, L.A., Rosenblum, D.S.: A historical perspective on runtime assertion checking in software development. ACM SIGSOFT Softw. Eng. Notes 31(3), 25–37 (2006). https://doi.org/10.1145/1127878.1127900

    Article  Google Scholar 

  17. Coppit, D., Haddox-Schatz, J.: On the use of specification-based assertions as test oracles. In: IEEE/NASA Software Engineering Workshop, pp. 305–314 (2005). https://doi.org/10.1109/SEW.2005.33, ISSN: 1550-6215

  18. Danglot, B., Vera-Perez, O., Baudry, B., Monperrus, M.: Automatic test improvement with DSpot: a study with ten mature open-source projects. Empir. Softw. Eng. 24(4), 2603–2635 (2019). https://doi.org/10.1007/s10664-019-09692-y

    Article  Google Scholar 

  19. Daniel, B., Gvero, T., Marinov, D.: On test repair using symbolic execution. In: ACM International Symposium on Software Testing and Analysis, pp. 207–218. ISSTA ’10, ACM (2010). https://doi.org/10.1145/1831708.1831734

  20. De Stefano, M., Pecorelli, F., Di Nucci, D., De Lucia, A.: A preliminary evaluation on the relationship among architectural and test smells. In: IEEE International Working Conference on Source Code Analysis and Manipulation, pp. 66–70 (2022). https://doi.org/10.1109/SCAM55253.2022.00013, ISSN: 2470-6892

  21. Delamare, R., Baudry, B., Ghosh, S., Le Traon, Y.: A test-driven approach to developing pointcut descriptors in AspectJ. In: IEEE International Conference on Software Testing Verification and Validation. IEEE Computing Society. (2009). https://doi.org/10.1109/ICST.2009.41

  22. Dinella, E., Ryan, G., Mytkowicz, T., Lahiri, S.K.: TOGA: a neural method for test oracle generation. In: IEEE/ACM International Conference on Software Engineering, pp. 2130–2141. ACM (2022). https://doi.org/10.1145/3510003.3510141

  23. Drusinsky, D., Shing, M.T., Demir, K.: Creation and validation of embedded assertion statecharts. In: IEEE International Workshop on Rapid System Prototyping, pp. 17–23 (2006). https://doi.org/10.1109/RSP.2006.12, ISSN: 1074-6005

  24. Dutta, S., Shi, A., Misailovic, S.: FLEX: fixing flaky tests in machine learning projects by updating assertion bounds. In: ACM Joint Meeting on European Software Engineering Conf. and Symposium on the Foundations of Software Engineering, pp. 603–614. ESEC/FSE 2021, ACM (2021). https://doi.org/10.1145/3468264.3468615

  25. Engström, E., Storey, M., Runeson, P., Höst, M., Baldassarre, M.T.: How software engineering research aligns with design science: a review. Empir. Softw. Eng. 25, 2630–2660 (2020). https://doi.org/10.1007/s10664-020-09818-7

    Article  Google Scholar 

  26. Franke, D., Kowalewski, S., Weise, C., Prakobkosol, N.: Testing conformance of life cycle dependent properties of mobile applications. In: IEEE International Conference on Software Testing, Verification and Validation, pp. 241–250 (2012). https://doi.org/10.1109/ICST.2012.104, ISSN: 2159-4848

  27. Fraser, G., Arcuri, A.: EvoSuite: automatic test suite generation for object-oriented software. In: ACM SIGSOFT Symposium and European Conference on Foundations of Software Engineering, pp. 416–419. ESEC/FSE ’11, ACM (2011). https://doi.org/10.1145/2025113.2025179

  28. Fraser, G., Zeller, A.: Mutation-driven generation of unit tests and oracles. IEEE Trans. Softw. Eng. 38(2), 278–292 (2012). https://doi.org/10.1109/TSE.2011.93

    Article  Google Scholar 

  29. Garousi, V., Amannejad, Y., Betin Can, A.: Software test-code engineering: a systematic mapping. Inf. Softw. Technol. 58, 123–147 (2015). https://doi.org/10.1016/j.infsof.2014.06.009

    Article  Google Scholar 

  30. Garousi, V., Felderer, M., Kılıçaslan, F.N.: A survey on software testability. Inf. Softw. Technol. 108, 35–64 (2019). https://doi.org/10.1016/j.infsof.2018.12.003

    Article  Google Scholar 

  31. Garousi, V., Felderer, M., Mäntylä, M.V.: Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Inf. Softw. Technol. 106, 101–121 (2019). https://doi.org/10.1016/j.infsof.2018.09.006

    Article  Google Scholar 

  32. Garousi, V., Mäntylä, M.V.: A systematic literature review of literature reviews in software testing. Inf. Softw. Technol. 80, 195–216 (2016). https://doi.org/10.1016/j.infsof.2016.09.002

    Article  Google Scholar 

  33. Pham, L.H., Tran Thi, L.L., Sun, J.: Assertion generation through active learning. In: Duan, Z., Ong, L. (eds.) Formal Methods and Software Engineering, pp. 174–191. LNCS, Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68690-5_11

  34. Jahangirova, G., Clark, D., Harman, M., Tonella, P.: OASIs: oracle assessment and improvement tool. In: ACM SIGSOFT International Symposium on Software Testing and Analysis. ISSTA 2018, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3213846.3229503

  35. Ji, T., Chen, L., Mao, X., Yi, X., Jiang, J.: Automated regression unit test generation for program merges (2020). http://arxiv.org/abs/2003.00154

  36. Kang, D., Raghavan, D., Bailis, P., Zaharia, M.: Model assertions for monitoring and improving ML models (2020). http://arxiv.org/abs/2003.01668

  37. Khurshid, S., Marinov, D.: TestEra: specification-based testing of Java programs using SAT. Autom. Softw. Eng. 11(4), 403–434 (2004). https://doi.org/10.1023/B:AUSE.0000038938.10589.b9

    Article  Google Scholar 

  38. Kitchenham, B.A., Budgen, D., Brereton, O.P.: Using mapping studies as the basis for further research – a participant-observer case study. Inf. Softw. Technol. 53(6), 638–651 (2011). https://doi.org/10.1016/j.infsof.2010.12.011

    Article  Google Scholar 

  39. Knauth, T., Fetzer, C., Felber, P.: Assertion-driven development: assessing the quality of contracts using meta-mutations. In: IEEE International Conference on Software Testing, Verification, and Validation Workshops, pp. 182–191 (2009). https://doi.org/10.1109/ICSTW.2009.40

  40. Koesnandar, A., Elbaum, S., Rothermel, G., Hochstein, L., Scaffidi, C., Stolee, K.T.: Using assertions to help end-user programmers create dependable web macros. In: ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 124–134. SIGSOFT ’08/FSE-16, ACM (2008). https://doi.org/10.1145/1453101.1453119

  41. Leotta, M., Cerioli, M., Olianas, D., Ricca, F.: Fluent vs basic assertions in Java: an empirical study. In: International Conference on the Quality of Information and Communications Technology (QUATIC), pp. 184–192 (2018). https://doi.org/10.1109/QUATIC.2018.00036

  42. Li, N., Offutt, J.: Test oracle strategies for model-based testing. IEEE Trans. Softw. Eng. 43(4), 372–395 (2017). https://doi.org/10.1109/TSE.2016.2597136

    Article  Google Scholar 

  43. Li, P., Sun, J., Wang, H.: Formal approach to assertion-based code generation. Int. J. Softw. Eng. Knowl. Eng. 27(9), 1637–1662 (2017). https://doi.org/10.1142/S0218194017400162, publisher: World Scientific Publishing Co

  44. Loyola, P., Staats, M., Ko, I., Rothermel, G.: Dodona: automated oracle data set selection. In: ACM International Symposium on Software Testing and Analysis, pp. 193–203. ISSTA 2014, ACM (2014). https://doi.org/10.1145/2610384.2610408

  45. Ma’ayan, D.D.: The quality of Junit tests: an empirical study report. In: IEEE/ACM 1st International Workshop on Software Qualities and their Dependencies, pp. 33–36 (2018)

    Google Scholar 

  46. Martinez, M., Etien, A., Ducasse, S., Fuhrman, C.: RTj: a java framework for detecting and refactoring rotten green test cases. In: IEEE/ACM International Conference on Software Engineering: ICSE-Companion, pp. 69–72 (2020). publisher: ACM

    Google Scholar 

  47. Mayer, E.C.: Assertion-based testing of go programs. Master thesis, Technical University Munich (2020)

    Google Scholar 

  48. Mesbah, A., van Deursen, A., Roest, D.: Invariant-based automatic testing of modern web applications. IEEE Trans. Softw. Eng. 38(1), 35–53 (2012). https://doi.org/10.1109/TSE.2011.28

    Article  Google Scholar 

  49. Messaoudi, S., Shin, D., Panichella, A., Bianculli, D., Briand, L.C.: Log-based slicing for system-level test cases. In: Cadar, C., Zhang, X. (eds.) ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 517–528. ACM (2021). https://doi.org/10.1145/3460319.3464824

  50. Milani Fard, A., Mirzaaghaei, M., Mesbah, A.: Leveraging existing tests in automated test generation for web applications. In: ACM/IEEE International Conference on Automated Software Engineering, pp. 67–78. ASE ’14, ACM (2014). https://doi.org/10.1145/2642937.2642991

  51. Mirshokraie, S., Mesbah, A., Pattabiraman, K.: Atrina: inferring unit oracles from GUI test cases. In: IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE Computer Society (2016). https://doi.org/10.1109/ICST.2016.32

  52. Molina, F., Ponzio, P., Aguirre, N., Frias, M.: EvoSpex: an evolutionary algorithm for learning postconditions (artifact). In: IEEE/ACM International Conference on Software Engineering: ICSE-Companion, pp. 185–186 (2021). https://doi.org/10.1109/ICSE-Companion52605.2021.00080, iSSN: 2574-1926

  53. Moonen, L., van Deursen, A., Zaidman, A., Bruntink, M.: On the interplay between software testing and evolution and its effect on program comprehension. In: Software Evolution, pp. 173–202. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-76440-3_8

    Chapter  Google Scholar 

  54. Murphy, C., Shen, K., Kaiser, G.: Using JML runtime assertion checking to automate metamorphic testing in applications without test oracles. In: IEEE International Conference on Software Testing Verification and Validation, pp. 436–445 (2009). https://doi.org/10.1109/ICST.2009.19, ISSN: 2159-4848

  55. Oliveira, R.A.P., Kanewala, U., Nardi, P.A.: Chapter three - automated test oracles: state of the art, taxonomies, and trends. In: Memon, A. (ed.) Advances in Computers, vol. 95, pp. 113–199. Elsevier (2014). https://doi.org/10.1016/B978-0-12-800160-8.00003-6

  56. Oriat, C.: Jartege: a tool for random generation of unit tests for Java classes. In: Reussner, R., Mayer, J., Stafford, J.A., Overhage, S., Becker, S., Schroeder, P.J. (eds.) QoSA/SOQUA -2005. LNCS, vol. 3712, pp. 242–256. Springer, Heidelberg (2005). https://doi.org/10.1007/11558569_18

    Chapter  Google Scholar 

  57. Pariente, D., Signoles, J.: Static analysis and runtime-assertion checking: contribution to security counter-measures (2017). https://zenodo.org/record/820856

  58. Pastore, F., Mariani, L., Fraser, G.: CrowdOracles: can the crowd solve the oracle problem? In: IEEE International Conference on Software Testing, Verification and Validation, pp. 342–351 (2013). https://doi.org/10.1109/ICST.2013.13, publisher: IEEE

  59. Patel, K., Hierons, R.M.: A mapping study on testing non-testable systems. Softw. Qual. J. 26(4), 1373–1413 (2018). https://doi.org/10.1007/s11219-017-9392-4

    Article  Google Scholar 

  60. Petke, J., Blot, A.: Refining fitness functions in test-based program repair. In: IEEE/ACM International Conference on Software Engineering Workshops, pp. 13–14. ICSEW’20, ACM (2020). https://doi.org/10.1145/3387940.3392180

  61. Pezzè, M., Zhang, C.: Chapter one - automated test oracles: a survey. In: Memon, A. (ed.) Advances in Computers, vol. 95, pp. 1–48. Elsevier (2014). https://doi.org/10.1016/B978-0-12-800160-8.00001-2

  62. Rajan, A., du Bousquet, L., Ledru, Y., Vega, G., Richier, J.L.: Assertion-based test oracles for home automation systems. In: ACM Int. Workshop on Model-Based Methodologies for Pervasive and Embedded Software, pp. 45–52. MOMPES ’10, ACM (2010). https://doi.org/10.1145/1865875.1865882

  63. Sakakibara, M., Sakurai, K., Komiya, S.: An assertion mechanism for software unit testing to remain unaffected by program modification - the mechanism to eliminate dependency from/to unnecessary object. Knowl.-Based Softw. Eng., pp. 125–134 (2008). https://doi.org/10.3233/978-1-58603-900-4-125

  64. Salehi Fathabadi, A., Dalvandi, M., Butler, M., Al-Hashimi, B.M.: Verifying cross-layer interactions through formal model-based assertion generation. IEEE Embed. Syst. Lett. 12(3), 83–86 (2020). https://doi.org/10.1109/LES.2019.2955316

    Article  Google Scholar 

  65. Santana, R., et al.: RAIDE: a tool for assertion roulette and duplicate assert identification and refactoring. In: Brazilian Symposium on Software Engineering, pp. 374–379. SBES ’20, ACM (2020). https://doi.org/10.1145/3422392.3422510

  66. Sequeira, S.: Understanding web application test assertion failures. Ph.D. thesis, University of British Columbia (2014). https://doi.org/10.14288/1.0167024

  67. Shamshiri, S., Just, R., Rojas, J.M., Fraser, G., McMinn, P., Arcuri, A.: Do automatically generated unit tests find real faults? an empirical study of effectiveness and challenges (t). In: IEEE/ACM International Conference on Automated Software Engineering, pp. 201–211 (2015). https://doi.org/10.1109/ASE.2015.86

  68. Shrestha, K., Rutherford, M.J.: An empirical evaluation of assertions as oracles. In: IEEE International Conference on Software Testing, Verification and Validation, pp. 110–119 (2011). https://doi.org/10.1109/ICST.2011.50, ISSN: 2159-4848

  69. Snyder, H.: Literature review as a research methodology: an overview and guidelines. J. Bus. Res. 104, 333–339 (2019). https://doi.org/10.1016/j.jbusres.2019.07.039

    Article  Google Scholar 

  70. Soares, E., Ribeiro, M., Gheyi, R., Amaral, G., Santos, A.: Refactoring test smells with JUnit 5: why should developers keep up-to-date? IEEE Trans. Softw. Eng. 49(3), 1152–1170 (2023). https://doi.org/10.1109/TSE.2022.3172654

    Article  Google Scholar 

  71. Spadini, D., Palomba, F., Zaidman, A., Bruntink, M., Bacchelli, A.: On the relation of test smells to software code quality. In: IEEE Intetnational Conference on Software Maintenance and Evolution, pp. 1–12 (2018). https://doi.org/10.1109/ICSME.2018.00010, ISSN: 2576-3148

  72. Stoyanova, V., Petrova-Antonova, D., Ilieva, S.: Automation of test case generation and execution for testing web service orchestrations. In: IEEE International Symposium on Service-Oriented System Engineering, pp. 274–279 (2013). https://doi.org/10.1109/SOSE.2013.9

  73. Søndergaard, H., Korsholm, S., Ravn, A.: Conformance test development with the Java modeling language. Concurr. Comput.: Pract. Exper. 29(22), 32 (2017). https://doi.org/10.1002/cpe.4071

  74. Terragni, V., Jahangirova, G., Tonella, P., Pezzè, M.: GAssert: a fully automated tool to improve assertion oracles. In: IEEE/ACM International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 85–88 (2021). https://doi.org/10.1109/ICSE-Companion52605.2021.00042, iSSN: 2574-1926

  75. Tiryaki, A.M., Öztuna, S., Dikenelli, O., Erdur, R.C.: SUNIT: a unit testing framework for test driven development of multi-agent systems. In: Padgham, L., Zambonelli, F. (eds.) AOSE 2006. LNCS, vol. 4405, pp. 156–173. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-70945-9_10

    Chapter  Google Scholar 

  76. Tsung-Hsiang, C., Yeh, T., Miller, R.C.: GUI testing using computer vision. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 1535–1544. ACM (2010). https://doi.org/10.1145/1753326.1753555

  77. Tufano, M., Drain, D., Svyatkovskiy, A., Sundaresan, N.: Generating accurate assert statements for unit test cases using pretrained transformers. In: ACM/IEEE International Conference on Automation of Software Test, pp. 54–64. AST ’22, ACM (2022). https://doi.org/10.1145/3524481.3527220

  78. Vahabzadeh, A., Milani Fard, A., Mesbah, A.: An empirical study of bugs in test code. In: IEEE International Conference on Software Maintenance and Evolution, pp. 101–110 (2015). https://doi.org/10.1109/ICSM.2015.7332456

  79. Valueian, M., Attar, N., Haghighi, H., Vahidi-Asl, M.: Constructing automated test oracle for low observable software. Scientia Iranica 27(3), 1333–1351 (2020). https://doi.org/10.24200/sci.2019.51494.2219

  80. Watson, C., Tufano, M., Moran, K., Bavota, G., Poshyvanyk, D.: On learning meaningful assert statements for unit test cases. In: ACM/IEEE International Conference on Software Engineering, pp. 1398–1409. ICSE ’20, ACM (2020). https://doi.org/10.1145/3377811.3380429

  81. Wei, C., Xiao, L., Yu, T., Chen, X., Wang, X., Wong, S., Clune, A.: Automatically tagging the “AAA” pattern in unit test cases using machine learning models. In: IEEE/ACM International Conference on Automated Software Engineering, pp. 1–3. ASE ’22, ACM (2023). https://doi.org/10.1145/3551349.3559510

  82. Weyuker, E.J.: On testing non-testable programs. Comput. J. 25(4), 465–470 (1982). https://doi.org/10.1093/comjnl/25.4.465

    Article  Google Scholar 

  83. Winkler, D., Urbanke, P., Ramler, R.: What do we know about readability of test code? - a systematic mapping study. In: IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 1167–1174 (2022). https://doi.org/10.1109/SANER53432.2022.00135, ISSN: 1534-5351

  84. Wu, H., Yin, R., Gao, J., Huang, Z., Huang, H.: To what extent can code quality be improved by eliminating test smells? In: International Conference on Code Quality, pp. 19–26 (2022). https://doi.org/10.1109/ICCQ53703.2022.9763153

  85. Xie, Q., Memon, A.M.: Designing and comparing automated test oracles for GUI-based software applications. ACM Trans. Softw. Eng. Methodol. 16(1), 4-es (2007). https://doi.org/10.1145/1189748.1189752

  86. Xie, T.: Augmenting automatically generated unit-test suites with regression oracle checking. In: Thomas, D. (ed.) ECOOP - Object-Oriented Programming, pp. 380–403. LNCS, Springer, Berlin, Heidelberg (2006). https://doi.org/10.1007/11785477_23

  87. Xie, T., Tillmann, N., de Halleux, J., Schulte, W.: Mutation analysis of parameterized unit tests. In: IEEE International Conference on Software Testing, Verification, and Validation Workshops, pp. 177–181 (2009). https://doi.org/10.1109/ICSTW.2009.43

  88. Xuan, J., Monperrus, M.: Test case purification for improving fault localization. In: ACM/SIGSOFT International Symposium on Foundations of Software Engineering, pp. 52–63. ACM (2014). https://doi.org/10.1145/2635868.2635906

  89. Xu, Y., Huang, B., Wu, G., Yuan, M.: Using genetic algorithms to repair JUnit test cases. In: Asia-Pacific Software Engineering Conference, vol. 1. IEEE Computer Society (2014). https://doi.org/10.1109/APSEC.2014.51

  90. Yu, H., et al.: Automated assertion generation via information retrieval and its integration with deep learning. In: IEEE/ACM International Conference on Software Engineering, pp. 163–174 (2022). https://doi.org/10.1145/3510003.3510149, publisher: ACM

  91. Zamprogno, L., Hall, B., Holmes, R., Atlee, J.M.: Dynamic human-in-the-loop assertion generation. IEEE Trans. Softw. Eng. 49(4), 2337–2351 (2023). https://doi.org/10.1109/TSE.2022.3217544

    Article  Google Scholar 

  92. Zeng, F., Deng, C., Yuan, Y.: Assertion-directed test case generation. In: World Congress on Software Engineering, pp. 41–45 (2012). https://doi.org/10.1109/WCSE.2012.16

  93. Zhang, Y., Mesbah, A.: Assertions are strongly correlated with test suite effectiveness. In: ACM Joint Meeting on Foundations of Software Engineering, pp. 214–224. ESEC/FSE 2015, ACM (2015). https://doi.org/10.1145/2786805.2786858

  94. Zhao, J., Harris, I.G.: Automatic assertion generation from natural language specifications using subtree analysis. In: Design, Automation Test in Europe Conference Exhibition (DATE), pp. 598–601 (2019). https://doi.org/10.23919/DATE.2019.8714857, iSSN: 1558–1101

  95. Zheng, F., Lam, P.: Identifying test refactoring candidates with assertion fingerprints. In: ACM Principles and Practices of Programming on The Java Platform, pp. 125–137. PPPJ ’15, ACM (2015). https://doi.org/10.1145/2807426.2807437

  96. Zhi, J., Garousi, V.: On adequacy of assertions in automated test suites: an empirical investigation. In: IEEE International Conference on Software Testing, Verification and Validation Workshops. pp. 382–391 (2013). https://doi.org/10.1109/ICSTW.2013.49

Download references

Acknowledgements

This work is funded by the ELLIIT strategic research area (https://elliit.se), project ‘A19 – Software Regression Testing with Near Failure Assertions’.

Author information

Authors and Affiliations

  1. Lund University, 221 00, Lund, Sweden

    Masoumeh Taromirad & Per Runeson

  2. Jönköping University, 551 11, Jönköping, Sweden

    Masoumeh Taromirad

Authors
  1. Masoumeh Taromirad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Per Runeson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masoumeh Taromirad .

Editor information

Editors and Affiliations

  1. Charles University, Praha, Czech Republic

    Jan Kofroň

  2. University of Limerick, Limerick, Ireland

    Tiziana Margaria

  3. Mälardalen University, Västerås, Sweden

    Cristina Seceleanu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Taromirad, M., Runeson, P. (2024). A Literature Survey of Assertions in Software Testing. In: Kofroň, J., Margaria, T., Seceleanu, C. (eds) Engineering of Computer-Based Systems. ECBS 2023. Lecture Notes in Computer Science, vol 14390. Springer, Cham. https://doi.org/10.1007/978-3-031-49252-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49252-5_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49251-8

  • Online ISBN: 978-3-031-49252-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics