skip to main content
10.1145/1621607.1621612acmconferencesArticle/Chapter ViewAbstractPublication PagesgpceConference Proceedingsconference-collections
research-article

The axioms strike back: testing with concepts and axioms in C++

Published: 04 October 2009 Publication History

Abstract

Modern development practises encourage extensive testing of code while it is still under development, using unit tests to check individual code units in isolation. Such tests are typically case-based, checking a likely error scenario or an error that has previously been identified and fixed. Coming up with good test cases is challenging, and focusing on individual tests can distract from creating tests that cover the full functionality.
Axioms, known from program specification, allow for an alternative way of generating test cases, where the intended functionality is described as rules or equations that can be checked automatically. Axioms are proposed as part of the concept feature of the upcoming C++0x standard.
In this paper, we describe how tests may be generated automatically from axioms in C++ concepts, and supplied with appropriate test data to form effective automated unit tests.

References

[1]
S. Antoy. Systematic design of algebraic specifications. In IWSSD '89: Proceedings of the 5th international workshop on Software specification and design, pages 278--280. ACM, New York, NY, USA, 1989. ISBN0-89791-305-1.
[2]
S. Antoy and R. G. Hamlet. Automatically checking an implementation against its formal specification. IEEE Trans. Software Eng., 26 (1): 55--69, 2000.
[3]
A. H. Bagge, V. David, and M. Haveraaen. Axiom-based testing for C++. In OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 721--722. ACM, New York, NY, USA, 2008. ISBN 978-1-60558-220-7.
[4]
A. H. Bagge and M. Haveraaen. Axiom-based transformations: Optimisation and testing. In J. J. Vinju and A. Johnstone, editors, Eighth Workshop on Language Descriptions, Tools and Applications (LDTA 2008), Electronic Notes in Theoretical Computer Science. Elsevier, Budapest, Hungary, 2008.
[5]
A. H. Bagge and M. Haveraaen. Interfacing concepts: Why declaration style shouldn't matter. In T. Ekman and J. J. Vinju, editors, Proceedings of the Ninth Workshop on Language Descriptions, Tools and Applications (LDTA '09), Electronic Notes in Theoretical Computer Science. Elsevier, York, UK, March 2009.
[6]
K. Beck. Extreme programming: A humanistic discipline of software development. In Proceedings of the 1st International Conference on Fundamental Approaches to Software Engineering (FASE'98), pages 1--6. Springer Berlin / Heidelberg, 1998.
[7]
K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2002. ISBN 0321146530.
[8]
K. Beck and E. Gamma. JUnit -- Java Unit testing. http://www.junit.org and http://junit.sourceforge.net/ per 2007-03-15.
[9]
P. Becker. Working draft, standard for programming language C++. Technical Report N2857=09-0047, JTC1/SC22/WG21 -- The C++ Standards Committee, March 23rd, 2009.
[10]
A. Borghi, V. David, and A. Demaille. C-Transformers: A framework to write C program transformations. ACM Crossroads, 12(3), April 2006.
[11]
M. Bravenboer, K. T. Kalleberg, R. Vermaas, and E. Visser. Stratego/XT 0.16: Components for transformation systems. In F. Tip and J. Hatcliff, editors, PEPM'06: Workshop on Partial Evaluation and Program Manipulation. ACM Press, January 2006.
[12]
H. Y. Chen, T. H. Tse, F. T. Chan, and T. Y. Chen. In black and white: an integrated approach to class-level testing of object-oriented programs. ACM Trans. Softw. Eng. Methodol., 7(3):250--295, 1998. ISSN 1049-331X.
[13]
H. Y. Chen, T. H. Tse, and T. Y. Chen. Taccle: a methodology for object-oriented software testing at the class and cluster levels. ACM Trans. Softw. Eng. Methodol., 10(1):56--109, 2001. ISSN 1049-331X.
[14]
K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP '00: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming, pages 268--279. ACM, New York, NY, USA, 2000. ISBN 1-58113-202-6.
[15]
R.-K. Doong. An approach to testing object-oriented programs. PhD thesis, Polytechnic University, Brooklyn, NY, USA, 1993.
[16]
R.-K. Doong and P. G. Frankl. Case studies on testing object-oriented programs. In TAV4: Proceedings of the symposium on Testing, analysis, and verification, pages 165--177. ACM Press, New York, NY, USA, 1991. ISBN 0-89791-449-X.
[17]
R.-K. Doong and P. G. Frankl. The ASTOOT approach to testing object-oriented programs. ACM Trans. Softw. Eng. Methodol., 3(2): 101--130, 1994. ISSN 1049-331X.
[18]
J. Gannon, P. McMullin, and R. Hamlet. Data abstraction, implementation, specification, and testing. ACM Trans. Program. Lang. Syst., 3(3):211--223, 1981. ISSN 0164-0925.
[19]
M.-C. Gaudel. Testing can be formal, too. In P. D. Mosses, M. Nielsen, and M. I. Schwartzbach, editors, TAPSOFT, volume 915 of Lecture Notes in Computer Science, pages 82--96. Springer, 1995. ISBN 3-540-59293-8.
[20]
M.-C. Gaudel and P. L. Gall. Testing data types implementations from algebraic specifications. In Formal Methods and Testing, volume 4949 of Lecture Notes in Computer Science, pages 209--239. Springer, 2008.
[21]
J. Goguen, J. Thatcher, and E. Wagner. An initial algebra approach to the specification, correctness and implementation of abstract data types. In R. Yeh, editor, Current Trends in Programming Methodology, volume 4, pages 80--149. Prentice Hall, 1978.
[22]
P. Gottschling. Fundamental algebraic concepts in concept-enabled C++. Technical Report TR639, Department of Computer Science, Indiana University, 2006.
[23]
D. Gregor, J. Järvi, J. Siek, B. Stroustrup, G. D. Reis, and A. Lumsdaine. Concepts: linguistic support for generic programming in C++. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 291--310. ACM, New York, NY, USA, 2006. ISBN 1-59593-348-4.
[24]
D. Gregor and A. Lumsdaine. Core concepts for the C++0x standard library (revision 2). Technical Report N2621=08-0131, JTC1/SC22/WG21 -- The C++ Standards Committee, May 19th, 2008.
[25]
W. J. Gutjahr. Partition testing vs. random testing: The influence of uncertainty. IEEE Transactions on Software Engineering, 25(5): 661--674, 1999. ISSN 0098-5589.
[26]
J. V. Guttag. Notes on type abstraction (version 2). IEEE Trans. Softw. Eng., 6(1):13--23, 1980. ISSN 0098-5589.
[27]
J. V. Guttag and J. J. Horning. The algebraic specification of abstract data types. Acta Inf., 10:27--52, 1978.
[28]
J. V. Guttag, E. Horowitz, and D. R. Musser. Abstract data types and software validation. Commun. ACM, 21(12):1048--1064, 1978. ISSN 0001-0782.
[29]
D. Hamlet and R. Taylor. Partition testing does not inspire confidence. IEEE Trans. Softw. Eng., 16(12):1402--1411, 1990. ISSN 0098-5589.
[30]
R. Hamlet. Random testing. In J. Marciniak, editor, Encyclopedia of Software Engineering, pages 970--978. Wiley, 1994.
[31]
M. Haveraaen and E. Brkic. Structured testing in Sophus. In E. Coward, editor, Norsk informatikkonferanse NIK'2005, pages 43--54. Tapir akademisk forlag, Trondheim, Norway, 2005. URL http://www.nik.no/2005/.
[32]
M. Haveraaen, H. A. Friis, and H. Munthe-Kaas. Computable scalar fields: a basis for PDE software. Journal of Logic and Algebraic Programming, 65(1):36--49, September-October 2005.
[33]
M. Haveraaen and K. T. Kalleberg. JAxT and JDI: the simplicity of JUnit applied to axioms and data invariants. In OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 731--732. ACM, New York, NY, USA, 2008. ISBN 978-1-60558-220-7.
[34]
M. Hughes and D. Stotts. Daistish: systematic algebraic testing for OO programs in the presence of side-effects. In ISSTA '96: Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis, pages 53--61. ACM Press, New York, NY, USA, 1996. ISBN 0-89791-787-1.
[35]
K. T. Kalleberg. Abstractions for Language-Independent Program Transformations, chapter 11. University of Bergen, Norway, Postboks 7800, 5020 Bergen, Norway, June 2007. ISBN 978-82-308-0441-4.
[36]
B. Liskov and S. Zilles. Specification techniques for data abstractions. In Proceedings of the international conference on Reliable software, pages 72--87. ACM, New York, NY, USA, 1975.
[37]
B. Liskov and S. Zilles. Specification techniques for data abstractions. In Proceedings of the international conference on Reliable software, pages 72--87. ACM, New York, NY, USA, 1975.
[38]
M. Masood, E. Birkenes, K. T. Kalleberg, M. Haveraaen, and A. H. Bagge. Axiom-based testing of Java collections with JAxT. Technical Report 388, Department of Informatics, University of Bergen, P.O.Box 7803, N-5020 Bergen, Norway, August 2009. URL http://www.ii.uib.no/publikasjoner/texrap/.
[39]
I. S. W. B. Prasetya, T. E. J. Vos, and A. Baars. Trace-based reflexive testing of OO programs. Technical Report UU-CS-2007-037, Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands, 2007. URL http://www.cs.uu.nl/research/techreps/repo/CS-2007/2007-037.pdf.
[40]
D. Saff. Theory-infected: or how I learned to stop worrying and love universal quantification. In OOPSLA '07: Companion to the 22nd ACM SIGPLAN conference on Object oriented programming systems and applications companion, pages 846--847. ACM, New York, NY, USA, 2007. ISBN 978-1-59593-865-7.
[41]
K. Sen. Effective random testing of concurrent programs. In ASE'07: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, pages 323--332. ACM, New York, NY, USA, 2007. ISBN 978-1-59593-882-4.
[42]
J. G. Siek and A. Lumsdaine. The matrix template library: A unifying framework for numerical linear algebra. In ECOOP '98: Workshop on Object-Oriented Technology, pages 466--467. Springer-Verlag, London, UK, 1998. ISBN 3-540-65460-7.
[43]
P. Sommerlad. C++ Unit Testing Easier, 2009. URL http://r2.ifs.hsr.ch/cute.
[44]
P. D. Stotts, M. Lindsey, and A. Antley. An informal formal method for systematic JUnit test case generation. In D. Wells and L. A. Williams, editors, XP/Agile Universe, volume 2418 of Lecture Notes in Computer Science, pages 131--143. Springer, 2002. ISBN 3-540-44024-0.
[45]
X. Tang and J. Järvi. Concept-based optimization. In LCSD '07: Proceedings of the 2007 Symposium on Library-Centric Software Design, pages 97--108. ACM, New York, NY, USA, 2007. ISBN 978-1-60558-086-9.
[46]
The Transformers Group, LRDE, EPITA. Transformers, 2008. URL http://www.lrde.epita.fr/cgi-bin/twiki/view/Transformers/Transformers.
[47]
B. Yu, L. Kong, Y. Zhang, and H. Zhu. Testing java components based on algebraic specifications. In ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pages 190--199. IEEE Computer Society, Washington, DC, USA, 2008. ISBN 978-0-7695-3127-4.
[48]
H. Zhu. A note on test oracles and semantics of algebraic specifications. In QSIC '03: Proceedings of the Third International Conference on Quality Software, pages 91--98. IEEE Computer Society, Washington, DC, USA, 2003. ISBN 0-7695-2015-4.

Cited By

View all
  • (2012)Testing C++ generic librariesProceedings of the 2012 IEEE International Conference on Software Maintenance (ICSM)10.1109/ICSM.2012.6405251(36-45)Online publication date: 23-Sep-2012
  • (2011)Design of concept libraries for c++Proceedings of the 4th international conference on Software Language Engineering10.1007/978-3-642-28830-2_6(97-118)Online publication date: 3-Jul-2011

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GPCE '09: Proceedings of the eighth international conference on Generative programming and component engineering
October 2009
194 pages
ISBN:9781605584942
DOI:10.1145/1621607
  • General Chair:
  • Jeremy Siek,
  • Program Chair:
  • Bernd Fischer
  • cover image ACM SIGPLAN Notices
    ACM SIGPLAN Notices  Volume 45, Issue 2
    GPCE '09
    February 2010
    182 pages
    ISSN:0362-1340
    EISSN:1558-1160
    DOI:10.1145/1837852
    Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. C++
  2. C++0x
  3. algebraic specification
  4. axiom-based testing
  5. axioms
  6. concepts
  7. generative programming
  8. mouldable programming
  9. program transformation
  10. test generation
  11. unit testing

Qualifiers

  • Research-article

Conference

GPCE'09
Sponsor:
GPCE'09: Generative Programming and Component Engineering
October 4 - 5, 2009
Colorado, Denver, USA

Acceptance Rates

GPCE '09 Paper Acceptance Rate 18 of 62 submissions, 29%;
Overall Acceptance Rate 56 of 180 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2012)Testing C++ generic librariesProceedings of the 2012 IEEE International Conference on Software Maintenance (ICSM)10.1109/ICSM.2012.6405251(36-45)Online publication date: 23-Sep-2012
  • (2011)Design of concept libraries for c++Proceedings of the 4th international conference on Software Language Engineering10.1007/978-3-642-28830-2_6(97-118)Online publication date: 3-Jul-2011

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media