research-article

Towards efficient measuring of web services API coverage

Authors:

Waldemar Hummer,

Schahram DustdarAuthors Info & Claims

PESOS '11: Proceedings of the 3rd International Workshop on Principles of Engineering Service-Oriented Systems

Pages 22 - 28

https://doi.org/10.1145/1985394.1985398

Published: 23 May 2011 Publication History

Abstract

We address the problem of interface-based test coverage for Web services. We suggest an approach to analyze the Application Programming Interface (API) of Web services, calculate the number of possible input combinations and compare it to the number of actual historical invocations. Such API coverage metrics are an indicator to which extent the service has been used. Measuring API coverage is a key concern for assessing the significance of Verification and Validation (V&V) techniques; on the other hand, API coverage metrics can also yield interesting usage reports for a service-based system in production use. The coverage metrics rely on the exact specification of service interfaces, and we provide a mechanism to specify restrictions for data types in the Java Web services framework (JAX-WS). As full enumeration of all possible inputs is often infeasible, we allow the definition of custom coverage metrics by means of domain partitioning: the user divides domain ranges into subsets, and a coverage of 100% means that the logged invocations contain at least one sample for each subset. Based on a prototype implementation, we evaluate different aspects of our approach.

References

[1]

X. Bai, W. Dong, W.-T. Tsai, and Y. Chen. WSDL-based automatic test case generation for Web services testing. In Int. Workshop Service-Oriented Syst. Eng., pages 215--220, 2005.

Digital Library

[2]

X. Bai, Y. Wang, G. Dai, W.-T. Tsai, and Y. Chen. A framework for contract-based collaborative verification and validation of web services. In CBSE '10, pages 258--273, 2007.

Digital Library

[3]

L. Baresi and E. D. Nitto. Test and Analysis of Web Services. Springer-Verlag New York, Inc., 2007.

Digital Library

[4]

C. Bartolini, A. Bertolino, S. Elbaum, and E. Marchetti. Whitening SOA Testing. In ESEC/SIGSOFT FSE '09, 2009.

Digital Library

[5]

C. Bartolini, A. Bertolino, E. Marchetti, and A. Polini. WS-TAXI: A WSDL-based Testing Tool for Web Services. In ICST 2009, pages 326--335, 2009.

Digital Library

[6]

B. Beizer. Software testing techniques (2nd ed.). Van Nostrand Reinhold Co., New York, USA, 1990.

Digital Library

[7]

A. Bertolino, J. Gao, E. Marchetti, and A. Polini. Automatic Test Data Generation for XML Schema-based Partition Testing. In Int. Workshop Automation of Software Test, 2007.

Digital Library

[8]

R. V. Binder. Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman, 1999.

Digital Library

[9]

G. Canfora and M. Di Penta. Testing Services and Service-Centric Systems: Challenges and Opportunities. IT Professional, 8(2):10--17, 2006.

Digital Library

[10]

J. Garcïa-fanjul, J. Tuya, and C. D. L. Riva. Generating Test Cases Specifications for BPEL Compositions of Web Services Using SPIN. In WS-MaTe 2006, pages 83--94, 2006.

[11]

W. Hummer, O. Raz, O. Shehory, P. Leitner, and S. Dustdar. Test coverage of data-centric dynamic compositions in service-based systems. In 4th IEEE International Conference on Software Testing, Verification and Validation, 2011.

Digital Library

[12]

A. Jorgensen and J. Whittaker. An API Testing Method. In STAREAST Conf. on Softw. Testing Analysis & Review, 2000.

[13]

D. Lübke, L. Singer, and A. Salnikow. Calculating BPEL Test Coverage Through Instrumentation. In Int. Workshop on Automation of Software Test, pages 115--122, 2009.

[14]

L. Mei, W. Chan, and T. Tse. Data flow testing of service-oriented workflow applications. In ICSE, 2008.

Digital Library

[15]

M. P. Papazoglou, P. Traverso, S. Dustdar, and F. Leymann. Service-Oriented Computing: State of the Art and Research Challenges. Computer, 40(11):38--45, 2007.

Digital Library

[16]

J. O. Paul Ammann. Introduction to Software Testing. Cambridge University Press, 2008.

Digital Library

[17]

W. T. Tsai, Y. Chen, R. Paul, N. Liao, and H. Huang. Cooperative and Group Testing in Verification of Dynamic Composite Web Services. In COMPSAC, pages 170--173, 2004.

Digital Library

[18]

World Wide Web Consortium (W3C). XML Path Language (XPath). http://www.w3.org/TR/xpath/, 1999.

[19]

W. Xu, J. Offutt, and J. Luo. Testing Web Services by XML Perturbation. In 16th IEEE Int. Symposium on Software Reliability Engineering, pages 257--266, 2005.

Digital Library

Cited By

Gambi AHummer WDustdar SPaige RHarman MWilliams J(2013)Testing elastic systems with surrogate modelsProceedings of the 1st International Workshop on Combining Modelling and Search-Based Software Engineering10.5555/2662572.2662577(8-11)Online publication date: 19-May-2013
https://dl.acm.org/doi/10.5555/2662572.2662577
Wotawa FSchulz MPill IJehan SLeitner PHummer WSchulte SHoenisch PDustdar S(2013)Fifty Shades of Grey in SOA TestingProceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops10.1109/ICSTW.2013.26(154-157)Online publication date: 18-Mar-2013
https://dl.acm.org/doi/10.1109/ICSTW.2013.26
Gambi AHummer WDustdar S(2013)Testing elastic systems with surrogate models2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering (CMSBSE)10.1109/CMSBSE.2013.6604429(8-11)Online publication date: May-2013
https://doi.org/10.1109/CMSBSE.2013.6604429
Show More Cited By

Index Terms

Towards efficient measuring of web services API coverage
1. Information systems
  1. World Wide Web
    1. Web applications
    2. Web services

Recommendations

Test coverage criteria for RESTful web APIs
A-TEST 2019: Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

Web APIs following the REST architectural style (so-called RESTful web APIs) have become the de-facto standard for software integration. As RESTful APIs gain momentum, so does the testing of them. However, there is a lack of mechanisms to assess the ...
Composing Web Services: A QoS View

An Internet application can invoke several services--a stock-trading Web service, for example, could invoke a payment service, which could then invoke an authentication service. Such a scenario is called a composite Web service, and it can be specified ...
Metamorphic Testing for Web Services: Framework and a Case Study
ICWS '11: Proceedings of the 2011 IEEE International Conference on Web Services

Service Oriented Architecture (SOA) has become a major application development paradigm. As a basic unit of SOA applications, Web services significantly affect the quality of the applications constructed from them. Since the development and consumption ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PESOS '11: Proceedings of the 3rd International Workshop on Principles of Engineering Service-Oriented Systems

May 2011

68 pages

ISBN:9781450305914

DOI:10.1145/1985394

Program Chairs:
Manuel Carro
School of Computer Science, Technical University of Madrid, Spain
,
Dimka Karastoyanova
Institute of Architecture of Application Systems (IAAS), University of Stuttgart, Germany
,
Grace Lewis
Software Engineering Institute, Carnegie Mellon University, U.S.A.
,
Anna Liu
NICTA, Australia

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE11

Sponsor:

SIGSOFT

ICSE11: International Conference on Software Engineering

May 23 - 24, 2011

HI, Waikiki, Honolulu, USA

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
199
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gambi AHummer WDustdar SPaige RHarman MWilliams J(2013)Testing elastic systems with surrogate modelsProceedings of the 1st International Workshop on Combining Modelling and Search-Based Software Engineering10.5555/2662572.2662577(8-11)Online publication date: 19-May-2013
https://dl.acm.org/doi/10.5555/2662572.2662577
Wotawa FSchulz MPill IJehan SLeitner PHummer WSchulte SHoenisch PDustdar S(2013)Fifty Shades of Grey in SOA TestingProceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops10.1109/ICSTW.2013.26(154-157)Online publication date: 18-Mar-2013
https://dl.acm.org/doi/10.1109/ICSTW.2013.26
Gambi AHummer WDustdar S(2013)Testing elastic systems with surrogate models2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering (CMSBSE)10.1109/CMSBSE.2013.6604429(8-11)Online publication date: May-2013
https://doi.org/10.1109/CMSBSE.2013.6604429
Hummer WRaz OShehory OLeitner PDustdar S(2013)Testing of data‐centric and event‐based dynamic service compositionsSoftware Testing, Verification and Reliability10.1002/stvr.149323:6(465-497)Online publication date: 15-Apr-2013
https://doi.org/10.1002/stvr.1493

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten