skip to main content
10.1145/1114223.1114234acmotherconferencesArticle/Chapter ViewAbstractPublication PageswebmediaConference Proceedingsconference-collections
Article

Guidelines for performance evaluation of web services

Published: 05 December 2005 Publication History

Abstract

The integration of heterogeneous systems has been an argued problem. Web Service is a new technology and it provides flexibility and interconnection between different distributed applications over the Internet and intranets. Currently there are several Web Services implementations, which differ in their implementation language, support for class binding, ease of use and performance. This paper presents guidelines for performance evaluation of Web Services toolkits. The guidelines were developed based on known Web Services performance bottlenecks. These guidelines can be used to choose the most appropriate toolkit to develop a high performance-demanding service. Experiments were carried out to apply the guidelines, and measure and compare the performance of two mostly-used Java Web Services toolkits -- Apache Axis and Sun Java Web Services Developer Pack (JWSDP). In addition to performance guidelines, recommendations on the use of SOAP features to guarantee interoperability between toolkits are given.

References

[1]
Apache Software Foundation, "Web Services - Axis", http://ws.apache.org/axis.]]
[2]
Berners-Lee, T., Fielding, R. and Frystyk, H. Hypertext Transfer Protocol -- HTTP/1.0, IETF RFC 1945, http://www.ietf.org/rfc/rfc1945.]]
[3]
Box, D., Ehnebuske, D., Kakivaya, G., Layman, A., Mendelsohn, N., Nielsen, H., Thatte, S. and Winer, D. Simple Object Access Protocol (SOAP) 1.1, http://www.w3.org/TR/soap.]]
[4]
Chiu, K., Govindaraju, M. and Bramley, R. Investigating the Limits of SOAP Performance for Scientific Computing, In Procs. of the 11th IEEE International Symposium on High Performance Distributed Computing (HPDC 2002). IEEE Computer Society, Edinburgh, Scotland, 2002, 246--254.]]
[5]
F. Cohen, "Discover SOAP encoding's impact on Web service performance", http://www-128.ibm.com/developerworks/webservices/library/wssoapenc/.]]
[6]
Davis, D. and Parashar, M. Latency Performance of SOAP Implementations. In Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID). Berlin, Germany, 2002, 407--412.]]
[7]
Devaram, K. and Anresen, D. SOAP Optimization Via Parameterized Client-Side Caching. In Proceedings of the IASTED International Conference on Parallel and Distributed Computing and Systems. Marina Del Rey. CA, 2003, 785--790.]]
[8]
Elfwing, R., Paulsson, U. and Lundberg, L. Performance of SOAP in Web Service Environment Compared to CORBA. In Proceedings of the Ninth Asia-Pacific Software Engineering Conference (APSE). Queensland, Australia, 2002, 84.]]
[9]
van Engelen, R. Pushing the SOAP Envelope With Web Services for Scientific Computing, In Proceedings of the International Conference on Web Services (ICWS). Las Vegas, 2003, 346--352.]]
[10]
Ethereal, A Network Protocol Analyser, http://www.ethereal.com, 2004.]]
[11]
Govindaraju, M., Slominski, A., Chopella, V., Bramley, R. and Gannom, D. Requirements for and Evaluation of RMI Protocols for Scientific Computing, In Proceedings of the Conference on Supercomputing. Dallas, Texas, 2000.]]
[12]
Govindaraju, M. Slominski, A., Chiu, K., Liu, P., van Engelen, R. and Lewis, M. Toward Characterizing the Performance of SOAP Toolkits. In Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing. Pittsburg, USA, 2004, 365--372.]]
[13]
Juric, M., Kezmah, B., Hericko, M., Rozman, I. And Vezocnik, I. Java RMI, RMI Tunnelling and Web Services Comparison and Performance Analysis. ACM SIGPLAN Notice. Volume 39, 2004, 58--65.]]
[14]
Kohlhoff, C. and Steele, R. Evaluating SOAP for High Performance Business Applications: Real-Time Trading Systems. In Procs. of the 12th International World Wide Web Conference (WWW2003). Budapest, Hungary, 2003.]]
[15]
Manes, A. The wrapped document/literal convention, http://atmanes.blogspot.com/2005/03/wrappeddocumentliteral-convention.html, 2004.]]
[16]
Ng, A., Chen, S. and Greenfield, P. Evaluation of Contemporary Commercial SOAP, In Proceedings of the 5th Australasian Workshop on Software and System Architectures (AWSA). Melbourne, Australia, 2003, 64--71.]]
[17]
Roubtsov, V. My kingdom for a good timer, http://www.javaworld.com/javaworld/javaqa/2003-01/01-qa-0110-timing.html.]]
[18]
Shirasuma, S., Nakata, H., Matsuoka, S. and Sekiguchi, S. Evaluating Web Services Based Implementations of GridRPC. In Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing (HDPC 2002), Edinburgh, Scotland, 2002, 237--245.]]
[19]
Soap Ware.Org. The Leading Directory for SOAP 1.1 Developers, http://www.soapware.org/directory/4/implementations.]]
[20]
Sun Microsystems, Java Web Services Developer Pack (Java WSDP), http://java.sun.com/webservices/jwsdp/]]

Cited By

View all
  • (2022)Ethics: What is the Research Scenario in the Brazilian Symposium WebMedia?Proceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3539637.3557932(1-10)Online publication date: 7-Nov-2022
  • (2014)WebMedia XXProceedings of the 20th Brazilian Symposium on Multimedia and the Web10.1145/2664551.2664576(91-98)Online publication date: 18-Nov-2014
  • (2013)Real-Time Service-Oriented Architectures: A Data-Centric Implementation for Distributed and Heterogeneous Robotic SystemEmbedded Systems: Design, Analysis and Verification10.1007/978-3-642-38853-8_24(262-271)Online publication date: 2013
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
WebMedia '05: Proceedings of the 11th Brazilian Symposium on Multimedia and the web
December 2005
134 pages
ISBN:9781450378024
DOI:10.1145/1114223
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • SBC: Brazilian Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2005

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. SOAP
  2. XML
  3. guideline
  4. performance
  5. web services

Qualifiers

  • Article

Acceptance Rates

Overall Acceptance Rate 270 of 873 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)2
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Ethics: What is the Research Scenario in the Brazilian Symposium WebMedia?Proceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3539637.3557932(1-10)Online publication date: 7-Nov-2022
  • (2014)WebMedia XXProceedings of the 20th Brazilian Symposium on Multimedia and the Web10.1145/2664551.2664576(91-98)Online publication date: 18-Nov-2014
  • (2013)Real-Time Service-Oriented Architectures: A Data-Centric Implementation for Distributed and Heterogeneous Robotic SystemEmbedded Systems: Design, Analysis and Verification10.1007/978-3-642-38853-8_24(262-271)Online publication date: 2013
  • (2008)Web servicesProceedings of the 8th conference on Applied informatics and communications10.5555/1503829.1503849(115-119)Online publication date: 20-Aug-2008
  • (2008)Benchmark of middleware protocols for application and service interactionProceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia10.1145/1543137.1543145(40-47)Online publication date: 3-Dec-2008
  • (2006)JWSPerf: A Performance Benchmarking Utility with Support to Multiple Web Services ImplementationsAdvanced Int'l Conference on Telecommunications and Int'l Conference on Internet and Web Applications and Services (AICT-ICIW'06)10.1109/AICT-ICIW.2006.122(159-159)Online publication date: 2006

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media