research-article

Jsongen: a quickcheck based library for testing JSON web services

Authors:

Clara Benac Earle,

Lars-Åke Fredlund,

Ángel Herranz,

Julio MariñoAuthors Info & Claims

Erlang '14: Proceedings of the Thirteenth ACM SIGPLAN workshop on Erlang

Pages 33 - 41

https://doi.org/10.1145/2633448.2633454

Published: 03 September 2014 Publication History

Abstract

This article describes a systematic approach to testing behavioural aspects of Web Services that communicate using the JSON data format. As a key component, the Quviq QuickCheck property-based testing tool is used to automatically generate a large number of test cases from an abstract description of the service behaviour in the form of a finite state machine. The same behavioural description is also used to decide whether the execution of a test case is successful or not. To generate random JSON data for populating tests we have developed a new library, jsongen, which given a characterisation of the JSON data as a JSON schema, (i) automatically derives a QuickCheck generator which can generate an infinite number of JSON values that validate against the schema, and (ii) provides a generic QuickCheck state machine which is capable of following the (hyper)links documented in the JSON schema, to automatically explore the web service. The default behaviour of the state machine can be easily customized to include web service specific checks. The article illustrates the approach by developing a finite state machine model for the testing of a JSON-based web service.

References

[1]

T. Arts, J. Hughes, J. Johansson, and U. T. Wiger. Testing telecoms software with quviq quickcheck. In Proceedings of the 2006 ACM SIGPLAN Workshop on Erlang, pages 2--10, Portland, Oregon, USA, 2006.

Digital Library

[2]

X. Bai, W. Dong, W.-T. Tsai, and Y. Chen. Wsdl-based automatic test case generation for web services testing. In Service-Oriented System Engineering, 2005. SOSE 2005. IEEE International Workshop, pages 207--212, Oct 2005.

Digital Library

[3]

C. Bartolini, A. Bertolino, E. Marchetti, and A. Polini. Ws-taxi: A wsdl-based testing tool for web services. In Proceedings of the 2009 International Conference on Software Testing Verification and Validation, ICST '09, pages 326--335, Washington, DC, USA, 2009. IEEE Computer Society. ISBN 978-0-7695-3601-9.

Digital Library

[4]

K. Claessen, J. Duregård, and M. Palka. Generating constrained random data with uniform distribution. In Proceedings of the FLOPS 2014 conference, LNCS 8475. Springer, 2014.

[5]

C. B. Earle, L.-Å. Fredlund, Á. Herranz, and J. Mariño. Property-based testing of json based web services. In Proceedings 2014 IEEE Twenty-First International Conference on Web Services (ICWS 2014), July 2014.

[6]

ecma262. Standard ECMA-262. URL http://www.ecma-international.org/publications/standards/Ecma-262.htm.

[7]

M. A. Francisco, M. López, H. Ferreiro, and L. M. Castro. Turning web services descriptions into quickcheck models for automatic testing. In Proceedings of the Twelfth ACM SIGPLAN Workshop on Erlang, Erlang '13, pages 79--86, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2385-7.

Digital Library

[8]

ipsum. Schematic Ipsum. URL http://schematic-ipsum.herokuapp.com/.

[9]

jesse. JSON Schema Erlang (jesse). URL https://github.com/klarna/jesse.git.

[10]

json-pointers. JSON Pointer. URL http://tools.ietf.org/html/rfc6901.

[11]

jsongen source repository. jsongen. URL https://github.com/aherranz/jsongen.git.

[12]

L. Lampropoulos and K. F. Sagonas. Automatic wsdl-guided test case generation for proper testing of web services. In WWV'12, pages 3--16, 2012.

[13]

C. Runciman, M. Naylor, and F. Lindblad. Smallcheck and lazy smallcheck: automatic exhaustive testing for small values. In A. Gill, editor, Haskell, pages 37--48. ACM, 2008. ISBN 978-1-60558-064-7.

Digital Library

[14]

schema-random. JSON Schema random. URL https://github.com/andreineculau/json-schema-random.git.

[15]

Web JSON. JavaScript Object Notation. URL http://www.json.org/.

[16]

Web JSON-schema. JSON Schema. URL json-schema.org.

Cited By

Golmohammadi AZhang MArcuri A(2023)Testing RESTful APIs: A SurveyACM Transactions on Software Engineering and Methodology10.1145/361717533:1(1-41)Online publication date: 21-Aug-2023
https://dl.acm.org/doi/10.1145/3617175
Otto BKleinert T(2022)A Flow Graph based Approach for controlled Generation of AAS Digital Twin Instances for the Verification of Compliance Check ToolsIECON 2022 – 48th Annual Conference of the IEEE Industrial Electronics Society10.1109/IECON49645.2022.9968463(1-6)Online publication date: 17-Oct-2022
https://doi.org/10.1109/IECON49645.2022.9968463
Peng CGao YYang P(2022)Automated Server Testing: an Industrial Experience Report2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME55016.2022.00071(519-522)Online publication date: Oct-2022
https://doi.org/10.1109/ICSME55016.2022.00071
Show More Cited By

Index Terms

Jsongen: a quickcheck based library for testing JSON web services
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Foundations of JSON Schema
WWW '16: Proceedings of the 25th International Conference on World Wide Web

JSON -- the most popular data format for sending API requests and responses -- is still lacking a standardized schema or meta-data definition that allows the developers to specify the structure of JSON documents. JSON Schema is an attempt to provide a ...
Definition of REST web services with JSON schema

The Web has evolved from being a collection of documents to a collection of interconnected services that interoperate throughout the Internet. Web services are a natural consequence of this evolution. The web services model was initially centered around ...
JS4Geo: a canonical JSON Schema for geographic data suitable to NoSQL databases
Abstract
The large volume and variety of data produced in the current Big Data era lead companies to seek solutions for the efficient data management. Within this context, NoSQL databases rise as a better alternative to the traditional relational databases,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Erlang '14: Proceedings of the Thirteenth ACM SIGPLAN workshop on Erlang

September 2014

84 pages

ISBN:9781450330381

DOI:10.1145/2633448

General Chairs:
Laura M. Castro
Universidade da Coruña, Spain
,
Hans Svensson
Quviq AB, Sweden

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ICFP'14

Sponsor:

SIGPLAN

ICFP'14: ACM SIGPLAN International Conference on Functional Programming

September 5, 2014

Gothenburg, Sweden

Acceptance Rates

Erlang '14 Paper Acceptance Rate 9 of 14 submissions, 64%;

Overall Acceptance Rate 51 of 68 submissions, 75%

Upcoming Conference

ICFP '25

Sponsor:
sigplan

ACM SIGPLAN International Conference on Functional Programming

October 12 - 18, 2025

Singapore , Singapore

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
319
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Golmohammadi AZhang MArcuri A(2023)Testing RESTful APIs: A SurveyACM Transactions on Software Engineering and Methodology10.1145/361717533:1(1-41)Online publication date: 21-Aug-2023
https://dl.acm.org/doi/10.1145/3617175
Otto BKleinert T(2022)A Flow Graph based Approach for controlled Generation of AAS Digital Twin Instances for the Verification of Compliance Check ToolsIECON 2022 – 48th Annual Conference of the IEEE Industrial Electronics Society10.1109/IECON49645.2022.9968463(1-6)Online publication date: 17-Oct-2022
https://doi.org/10.1109/IECON49645.2022.9968463
Peng CGao YYang P(2022)Automated Server Testing: an Industrial Experience Report2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME55016.2022.00071(519-522)Online publication date: Oct-2022
https://doi.org/10.1109/ICSME55016.2022.00071
Karlsson SCausevic ASundmark D(2020)QuickREST: Property-based Test Generation of OpenAPI-Described RESTful APIs2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST46399.2020.00023(131-141)Online publication date: Oct-2020
https://doi.org/10.1109/ICST46399.2020.00023
Aichernig BSchumi R(2019)Property-based testing of web services by deriving properties from business-rule modelsSoftware and Systems Modeling (SoSyM)10.1007/s10270-017-0647-018:2(889-911)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1007/s10270-017-0647-0
Aichernig BSchumi R(2016)Property-Based Testing with FsCheck by Deriving Properties from Business Rule Models2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW.2016.24(219-228)Online publication date: Apr-2016
https://doi.org/10.1109/ICSTW.2016.24

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten