research-article

Random testing of purely functional abstract datatypes: guidelines for dealing with operation invariance

Author:

Stefan HoldermansAuthors Info & Claims

PPDP '13: Proceedings of the 15th Symposium on Principles and Practice of Declarative Programming

Pages 275 - 284

https://doi.org/10.1145/2505879.2505880

Published: 16 September 2013 Publication History

Abstract

Algebraic specification, equational reasoning, and property-based random testing provide functional programmers with a powerful yet reasonably lightweight framework for reasoning about abstract datatypes and assuring the quality of their concrete implementations. However, as it turns out, naïvely implementing property-based random testing for purely functional abstract datatypes is prone to subtle errors and may well leave unaware practitioners with a false sense of security about their datatype implementations. In this paper, we pinpoint one particular pitfall, namely overlooking the need to take into account the invariance of datatype operations under an implied equivalence relation on concreate datatype values, and discuss how to systematically avoid it. Presented in the context of a concrete case study, the proposed technique generalises nicely into a common design principle for engineering random tests of purely functional datastructures.

References

[1]

Thomas Arts, John Hughes, Joakim Johansson, and Ulf T. Wiger. Testing telecoms software with Quviq Quickcheck. In Marc Feeley and Philip W. Trinder, editors, Proceedings of the 2006 ACM SIGPLAN Workshop on Erlang, Portland, Oregon, USA, September 16, 2006, pages 2--10. ACM Press, 2006.

Digital Library

[2]

Thomas Arts, Laura M. Castro, and John Hughes. Testing Erlang data types with Quviq Quickcheck. In Soon Tee Teoh and Zoltán Horváth, editors, Proceedings of the 7th ACM SIGPLAN workshop on ERLANG, Victoria, BC, Canada, September 27, 2008, pages 1--8. ACM Press, 2008.

Digital Library

[3]

Jan A. Bergstra. Algebraic specification. ACM Press, New York, NY, USA, 1989.

[4]

Richard Bird. Introduction to Functional Programming using Haskell. Prentice Hall, London, England, UK, 2nd edition, 1998.

Digital Library

[5]

Manuel M. T. Chakravarty, Gabriele Keller, and Simon Peyton Jones. Associated type synonyms. In Olivier Danvy and Benjamin C. Pierce, editors, Proceedings of the 10th ACM SIGPLAN International Conference on Functional Programming, ICFP 2005, Tallinn, Estonia, September 26--28, 2005, pages 241--253. ACM Press, 2005.

Digital Library

[6]

Koen Claessen and John Hughes. QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming (ICFP '00), Montreal, Canada, September 18--21, 2000, pages 268--279. ACM Press, 2000.

Digital Library

[7]

Koen Claessen and John Hughes. Testing monadic code with QuickCheck. In Proceedings of the ACM SIGPLAN Workshop on Haskell, Pittsburgh, Pennsylvania, 2002, pages 65--77. ACM Press, 2002.

Digital Library

[8]

John D. Gannon, Paul R. McMullin, and Richard Hamlet. Data-abstraction implementation, specification, and testing. ACM Transactions on Programming Languages and Systems, 3(3):211--223, 1981.

Digital Library

[9]

Marie-Claude Gaudel and Pascale Le Gall. Testing data types implementations from algebraic specifications. In Robert M. Hierons, Jonathan P. Bown, and Mark Harman, editors, Formal Methods and Testing, An Outcome of the FORTEST Network, Revised Selected Papers, volume 4949 of Lecture Notes in Computer Science, pages 209--239. Springer-Verlag, 2008.

Digital Library

[10]

Andy Gill and Colin Runciman. Haskell program coverage. In Gabriele Keller, editor, Proceedings of the 2007 ACM SIGPLAN workshop on Haskell, Freiburg, Germany, September 30, 2007, pages 1--12. ACM Press, 2007.

Digital Library

[11]

John V. Guttag and James J. Horning. The algebraic specification of abstract data types. Acta Informatica, 10:27--52, 1978.

Digital Library

[12]

Richard Hamlet. Random testing. In John J. Marciniak, editor, Encyclopedia of Software Engineering, pages 970--978. Wiley, New York, NY, USA, 1994.

[13]

C. A. R. Hoare. Proof of correctness of data representations. Acta Informatica, 1:271--281, 1972.

Digital Library

[14]

Robert Hood and Robert Melville. Real-time queue operations in pure LISP. Information Processing Letters, 13(2): 50--54, 1981.

[15]

Johan Jeuring, Patrik Jansson, and Cláudio Amaral. Testing type class laws. In Janis Voigtländer, editor, Proceedings of the 5th ACM SIGPLAN Symposium on Haskell, Haskell 2012, Copenhagen, Denmark, 13 September 2012, pages 49--60. ACM Press, 2012.

Digital Library

[16]

Pieter W. M. Koopman, Artem Alimarine, Jan Tretmans, and Rinus Plasmeijer. Gast: Generic automated software testing. In Ricardo Peña and Thomas Arts, editors, Implementation of Functional Languages, 14th International Workshop, IFL 2002, Madrid, Spain, September 16--18, 2002, Revised Selected Papers, volume 2670 of Lecture Notes in Computer Science, pages 84--100. Springer-Verlag, 2003.

Digital Library

[17]

Pieter W. M. Koopman, Peter Achten, and Rinus Plasmeijer. Model based testing with logical properties versus state machines. In Andy Gill and Jurriaan Hage, editors, Implementation and Application of Functional Languages, 23rd International Symposium, IFL 2011, Lawrence, KS, USA, October 3--5, 2011, Revised Selected Papers, volume 7257 of Lecture Notes in Computer Science, pages 116--133. Springer-Verlag, 2012.

Digital Library

[18]

Barbara Liskov and Stephan N. Zilles. Programming with abstract data types. In Proceedings of the ACM SIGPLAN Symposium on Very High Level Languages, pages 50--59. ACM Press, 1974.

Digital Library

[19]

Robin Milner, Mads Tofte, Robin Harper, and David B. MacQueen. The Definition of Standard ML (Revised). The MIT Press, Cambridge, MA, USA, 1997.

Digital Library

[20]

Chris Okasaki. Purely Functional Data Structures. Cambridge University Press, Cambridge, England, U.K., 1999.

Digital Library

[21]

Simon Peyton Jones, editor. Haskell 98 Language and Libraries: The Revised Report. Cambridge University Press, Cambridge, England, U.K., 2003.

[22]

Rinus Plasmeijer and Marko van Eekelen. Concurrent Clean language report---version 1.3. Technical Report CSI-R9816, University of Nijmegen, 1998.

[23]

Philip Wadler. A critique of Abelson and Sussman or why calculating is better than scheming. SIGPLAN Notices, 22(3): 83--94, 1987.

Digital Library

[24]

Martin Wirsing. Algebraic specification. In Jan Van Leeuwen, editor, Handbook of Theoretical Computer Science, volume B: Formal Models and Semantics, pages 675--788. The MIT Press, Cambridge, MA, USA, 1990.

Digital Library

Cited By

Dongo ICardinale YChbeir R(2018)RDF-F: RDF Datatype inFerring FrameworkData Science and Engineering10.1007/s41019-018-0064-63:2(115-135)Online publication date: 11-Jun-2018
https://doi.org/10.1007/s41019-018-0064-6
Dongo ICardinale YAl-Khalil FChbeir R(2017)Semantic Web Datatype Inference: Towards Better RDF MatchingWeb Information Systems Engineering – WISE 201710.1007/978-3-319-68786-5_5(57-74)Online publication date: 4-Oct-2017
https://doi.org/10.1007/978-3-319-68786-5_5
Sergey I(2016)Experience report: growing and shrinking polygons for random testing of computational geometry algorithmsACM SIGPLAN Notices10.1145/3022670.295192751:9(193-199)Online publication date: 4-Sep-2016
https://dl.acm.org/doi/10.1145/3022670.2951927
Show More Cited By

Index Terms

Recommendations

ArbitCheck: A Highly Automated Property-Based Testing Tool for Java
ICSTW '14: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation Workshops

Lightweight property-based testing tools are becoming popular these days. With property-based testing, developers can test properties of the system under test against large varieties of randomly generated inputs without writing test cases. Despite the ...
Grammar-based testing for little languages: an experience report with student compilers
SLE 2020: Proceedings of the 13th ACM SIGPLAN International Conference on Software Language Engineering

We report on our experience in using various grammar-based test suite generation methods to test 61 single-pass compilers that undergraduate students submitted for the practical project of a computer architecture course.

We show that (1) all test ...
An Evaluation of Random Testing

Random testing of programs has usually (but not always) been viewed as a worst case of program testing. Testing strategies that take into account the program structure are generally preferred. Path testing is an often proposed ideal for structural ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PPDP '13: Proceedings of the 15th Symposium on Principles and Practice of Declarative Programming

September 2013

308 pages

ISBN:9781450321549

DOI:10.1145/2505879

General Chair:
Ricardo Peña
Universidad Complutense de Madrid
,
Program Chair:
Tom Schrijvers
Spain Ghent University, Belgium

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Universidad Complutense de Madrid

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PPDP '13

Sponsor:

PPDP '13: 15th International Symposium on Principles and Practice of Declarative Programming

September 16 - 18, 2013

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 230 of 486 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
66
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dongo ICardinale YChbeir R(2018)RDF-F: RDF Datatype inFerring FrameworkData Science and Engineering10.1007/s41019-018-0064-63:2(115-135)Online publication date: 11-Jun-2018
https://doi.org/10.1007/s41019-018-0064-6
Dongo ICardinale YAl-Khalil FChbeir R(2017)Semantic Web Datatype Inference: Towards Better RDF MatchingWeb Information Systems Engineering – WISE 201710.1007/978-3-319-68786-5_5(57-74)Online publication date: 4-Oct-2017
https://doi.org/10.1007/978-3-319-68786-5_5
Sergey I(2016)Experience report: growing and shrinking polygons for random testing of computational geometry algorithmsACM SIGPLAN Notices10.1145/3022670.295192751:9(193-199)Online publication date: 4-Sep-2016
https://dl.acm.org/doi/10.1145/3022670.2951927
Sergey IGarrigue JKeller GSumii E(2016)Experience report: growing and shrinking polygons for random testing of computational geometry algorithmsProceedings of the 21st ACM SIGPLAN International Conference on Functional Programming10.1145/2951913.2951927(193-199)Online publication date: 4-Sep-2016
https://dl.acm.org/doi/10.1145/2951913.2951927
Midtgaard JMoller A(2015)QuickChecking Static Analysis Properties2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST.2015.7102603(1-10)Online publication date: Apr-2015
https://doi.org/10.1109/ICST.2015.7102603

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten