Skip to main content
SpringerLink
Log in

Building Scalable Electronic Market Places Using HyperQuery-Based Distributed Query Processing

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Flexible distributed query processing capabilities are an important prerequisite for building scalable Internet applications, such as electronic Business-to-Business (B2B) market places. Architecting an electronic market place in a conventional data warehouse-like approach by integrating all the data from all participating enterprises in one centralized repository incurs severe problems: stale data, data security threats, administration overhead, inflexibility during query processing, etc. In this paper we present a new framework for dynamic distributed query processing based on so-called HyperQueries which are essentially query evaluation sub-plans “sitting behind” hyperlinks. Our approach facilitates the pre-materialization of static data at the market place whereas the dynamic data remains at the data sources. In contrast to traditional data integration systems, our approach executes essential (dynamic) parts of the data-integrating views at the data sources. The other, more static parts of the data are integrated à priori at the central portal, e.g., the market place. The portal serves as an intermediary between clients and data providers which execute their sub-queries referenced via hyperlinks. The hyperlinks are embedded as attribute values within data objects of the intermediary’s database. Retrieving such a virtual object will execute the referenced HyperQuery in order to materialize the missing data. We illustrate the flexibility of this distributed query processing architecture in the context of B2B electronic market places with an example derived from the car manufacturing industry.

Based on these HyperQueries, we propose a reference architecture for building scalable and dynamic electronic market places. All administrative tasks in such a distributed B2B market place are modeled as Web services and are initiated decentrally by the participants. Thus, sensitive data remains under the full control of the data providers. We describe optimization and implementation issues to obtain an efficient and highly flexible data integration platform for electronic market places. All proposed techniques have been fully implemented in our QueryFlow prototype system which served as the platform for our performance evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Abiteboul, B. Amann, J. Baumgarten, O. Benjelloun, A. Bonifati, G. Cobena, I. Manolescu, and T. Milo, “Active XML project,” http://www-rocq.inria.fr/gemo/Gemo/Projects/axml/index.html

  2. S. Abiteboul, A. Bonifati, G. Cobena, I. Manolescu, and T. Milo, “Dynamic XML documents with dis-tribution and replication,” in Proceedings of the ACM SIGMOD Conference on Management of Data, San Diego, CA, USA, 2003, pp. 527–538.

  3. Y. Arens, C. A. Knoblock, and W.-M. Shen, “Query reformulation for dynamic information integration,” Journal of Intelligent Information Systems 6(2–3), Special Issue on intelligent information integration, 1996, 99–130.

    Google Scholar 

  4. B. Atkinson, G. Della-Libera, S. Hada, M. Hondo, P. Hallam-Baker, C. Kaler, J. Klein, B. LaMacchia, P. Leach, J. Manferdelli, H. Maruyama, A. Nadalin, N. Nagaratnam, H. Prafullchandra, J. Shewchuk, and D. Simon, “Web service security (WS-Security),” http://msdn.microsoft.com/library/ default.asp?url=/library/en-us/dnglobspec/html/ws-security.asp, 2002.

  5. Y. Bakos, “A strategic analysis of electronic marketplaces,” MIS Quarterly 15(3), 1991, 295–310.

    Google Scholar 

  6. Y. Bakos, “The emerging role of electronic marketplaces on the Internet,” Communications of the ACM 41(8), 1998, 35–42.

    Google Scholar 

  7. M. Bartel, J. Boyer, B. Fox, B. LaMacchia, and E. Simon, “XML signature,” W3C Recommendation, http://www.w3.org/TR/xmldsig-core/, 2002.

  8. R. J. Bayardo, Jr., W. Bohrer, R. Brice, A. Cichocki, J. Fowler, A. Helal, V. Kashyap, T. Ksiezyk, G. Martin, M. Nodine, M. Rashid, M. Rusinkiewicz, R. Shea, C. Unnikrishnan, A. Unruh, and D. Woelk, “InfoSleuth: Agent-based semantic integration of information in open and dynamic environments,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Tucson, AZ, USA, 1997, pp. 195–206.

  9. BEA, “BEA WebLogic Enterprise Platform,” http://www.bea.com/framework.jsp?CNT= index.htm&FP=/content/solutions/te%chnical/platform

  10. P. Bernstein, N. Goodman, E. Wong, C. Reeve, and J. Rothnie, “Query processing in a system for distributed databases (SDD-1),” ACM Transactions on Database Systems 6(4), 1981, 602–625.

    Google Scholar 

  11. R. Braumandl, J. Claussen, A. Kemper, and D. Kossmann, “Functional join processing,” The VLDB Journal 8(3–4), Special Issue “Best of VLDB 98,” 2000, 156–177, invited contribution.

    Google Scholar 

  12. R. Braumandl, M. Keidl, A. Kemper, D. Kossmann, A. Kreutz, S. Seltzsam, and K. Stocker, “ObjectGlobe: Ubiquitous query processing on the Internet,” The VLDB Journal 10(3), Special Issue on e-services, 2001, 48–71.

    Google Scholar 

  13. I. Brunkhorst, H. Dhraief, A. Kemper, W. Nejdl, and C. Wiesner, “Distributed queries and query opti-mization in schema-based systems,” in International Workshop on Databases, Information Systems and Peer-to-Peer Computing, 2003.

  14. F. Casati, U. Dayal, and M.-C. Shan, “E-business application for supply chain management: challenges and solutions,” in Proceedings of the IEEE Conference on Data Engineering, Heidelberg, Germany, 2001, pp. 71–78.

  15. E. Christensen, F. Curbera, G. Meredith, and S. Weerawarana, “Web services description language (WSDL) 1.1,” W3C Note, http://www.w3.org/TR/wsdl, 2001.

  16. J. Claussen, A. Kemper, D. Kossmann, and C. Wiesner, “Exploiting early sorting and early partitioning for decision support query processing,” The VLDB Journal 9(3), Special Issue “Best of VLDB 99,” 2000, 190–213, invited contribution.

    Google Scholar 

  17. Covisint, http://www.covisint.com, 2000.

  18. V. Crescenzi, G. Mecca, and P. Merialdo, “RoadRunner: towards automatic data extraction from large Web sites,” in Proceedings of the Conference on Very Large Data Bases (VLDB), Rome, Italy, 2001, pp. 109–118.

  19. D. Draper, A. Y. Halevy, and D. S. Weld, “The nimble XML data integration system,” in Proceedings of the IEEE Conference on Data Engineering, Heidelberg, Germany, 2001, pp. 155–160.

  20. D. Florescu and D. Kossmann, “An XML programming language for Web service specification and composition,” IEEE Data Engineering Bulletin 24(2), 2001, 48–56.

    Google Scholar 

  21. R. Goldman and J. Widom, “WSQ/DSQ: a practical approach for combined querying of databases and the Web,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Dallas, TX, USA, 2000, pp. 285–296.

  22. G. Graefe, “Query evaluation techniques for large databases,” ACM Computing Surveys 25(2), 1993, 73–170.

    Google Scholar 

  23. L. Haas, D. Kossmann, E. Wimmers, and J. Yang, “Optimizing Queries across diverse data sources,” in Proceedings of the Conference on Very Large Data Bases (VLDB), Athens, Greece, 1997, pp. 276–285.

  24. A. Y. Halevy, Z. G. Ives, P. Mork, and I. Tatarinov, “Piazza: Data management infrastructure for semantic Web applications,” in International World Wide Web Conference, Budapest, Hungary, 2003.

  25. J. Hellerstein and J. Naughton, “Query execution strategies for caching expensive methods,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Montreal, Canada, 1996, pp. 423–434.

  26. J. M. Hellerstein, M. Stonebraker, and R. Caccia, “Independent, open enterprise data integration,” IEEE Data Engeneering Bulletin 22(1), 1999, 43–49.

    Google Scholar 

  27. R. Housley, W. Ford, W. Polk, and D. Solo, “Internet X.509 public key infrastructure certificate and CRL profile,” http://www.rfc-editor.org/rfc/rfc2459.txt, 1999.

  28. T. Imamura, B. Dillaway, and E. Simon, “XML encryption syntax and processing,” W3C Recommendation, http://www.w3.org/TR/xmlenc-core, 2002.

  29. Z. Ives, D. Florescu, M. Friedman, A. Levy, and D. Weld, “An adaptive query execution engine for data integration,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Philadelphia, PA, USA, 1999, pp. 299–310.

  30. A. Jhingran, “Moving up the food chain: supporting e-commerce applications on databases,” ACM SIG-MOD Record 29(4), 2000, 50–54.

    Google Scholar 

  31. V. Josifovski, P. Schwarz, L. Haas, and E. Lin, “Garlic: a new flavor of federated query processing for DB2,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Madison, WI, USA, 2002, pp. 524–532.

  32. M. Keidl, S. Seltzsam, K. Stocker, and A. Kemper, “ServiceGlobe: Distributing e-services across the Internet (demonstration),” in Proceedings of the Conference on Very Large Data Bases (VLDB), Hong Kong, China, 2002, pp. 1047–1050.

  33. T. Keller, G. Graefe, and D. Maier, “Efficient assembly of complex objects,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Denver, CO, USA, 1991, pp. 148–158.

  34. A. Kemper and C. Wiesner, “HyperQueries: dynamic distributed query processing on the Internet,” in Proceedings of the Conference on Very Large Data Bases (VLDB), Rome, Italy, 2001, pp. 551–560.

  35. C. A. Knoblock, S. Minton, J. L. Ambite, N. Ashish, I. Muslea, A. Philpot, and S. Tejada, “The Ariadne approach to Web-based information integration,” International Journal of Cooperative Information Systems 10(1–2), 2001, 145–169.

    Google Scholar 

  36. D. Kossmann, “The state of the art in distributed query processing,” ACM Computing Surveys 32(4), 2000, 422–469.

    Google Scholar 

  37. M. Lenzerini, “Data integration: A theoretical perspective,” in Proceedings of the ACM SIGACT-SIGMOD Symposium on Principles of Database Systems (PODS), Madison, WI, 2002, pp. 233–246.

  38. A. Levy, “Answering queries using views: a survey,” The VLDB Journal 10(4), 2001, 270–294.

    Google Scholar 

  39. A. Levy, A. Rajaraman, and J. Ordille, “Querying heterogeneous information sources using source descriptions,” in Proceedings of the Conference on Very Large Data Bases (VLDB), Bombay, India, 1996, pp. 251–262.

  40. A. Y. Levy, D. Srivastava, and T. Kirk, “Data model and query evaluation in global information systems,” Journal of Intelligent Information Systems (JIIS) 5(2), 1995, 121–143.

    Google Scholar 

  41. E. Mena, V. Kashyap, A. P. Sheth, and A. Illarramendi, “OBSERVER: An approach for query processing in global information systems based on interoperation across pre-existing ontologies,” in Conference on Cooperative Information Systems, Brussels, Belgium, 1996, pp. 14–25.

  42. Microsoft, “Microsoft.NET,” http://www.microsoft.com/net, 2001.

  43. W. Nejdl, M. Wolpers, W. Siberski, C. Schmitz, M. Schlosser, I. Brunkhorst, and A. Löser, “Super-peer-based routing and clustering strategies for RDF-based peer-to-peer networks,” in International World Wide Web Conference, Budapest, Hungary, 2003.

  44. M. H. Nodine, J. Fowler, T. Ksiezyk, B. Perry, M. Taylor, and A. Unruh, “Active information gathering in InfoSleuth,” International Journal of Cooperative Information Systems 9(1–2), 2000, 3–28.

    Google Scholar 

  45. S. Oaks, Java Security, O’Reilly, Sebastopol, CA, USA, 1998.

    Google Scholar 

  46. Y. Papakonstantinou, S. Abiteboul, and H. Garcia-Molina, “Object fusion in mediator systems,” in Proceedings of the Conference on Very Large Data Bases (VLDB), Bombay, India, 1996, pp. 413–424.

  47. Y. Papakonstantinou, A. Gupta, H. Garcia-Molina, and J. Ullman, “A query translation scheme for rapid implementation of wrappers,” in Proceedings of the Conference on Deductive and Object-Oriented Databases (DOOD), Singapore, 1995, pp. 161–186.

  48. E. Rahm and H.-H. Do, “Data cleaning: problems and current approaches,” IEEE Bulletin of the Technical Committee on Data Engineering 23(4), 2000, 3–13.

    Google Scholar 

  49. M. Rodriguez-Martinez and N. Roussopoulos, “MOCHA: A self-extensible database middleware system for distributed data sources,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Dallas, TX, USA, 2000, pp. 213–224.

  50. R. S. Sandhu, E. J. Coyne, H. L. Feinstein, and C. E. Youman, “Role-based access control models,” IEEE Computer 29(2), 1996, 38–47.

    Google Scholar 

  51. SAP, “Business Networking in the Internet Age,” Technical Report, SAP White Paper, http://www. sap-ag.de/germany/products/mysap/pdf/bus_networking.pdf, 1999.

  52. B. Schneider, Applied Cryptography, 2nd ed., Wiley, Chichester, UK, 1996.

    Google Scholar 

  53. J. Shanmugasundaram, J. Kiernan, E. J. Shekita, C. Fan, and J. Funderburk, “Querying XML views of relational data,” in Proceedings of the Conference on Very Large Data Bases (VLDB), Rome, Italy, 2001, pp. 261–270.

  54. J. Shanmugasundaram, E. J. Shekita, R. Barr, M. J. Carey, B. G. Lindsay, H. Pirahesh, and B. Reinwald, “Efficiently publishing relational data as XML documents,” The VLDB Journal 10(2–3), 2001, 133–154.

    Google Scholar 

  55. E. Shekita and M. Carey, “A performance evaluation of pointer-based joins,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Atlantic City, NJ, USA, 1990, pp. 300–311.

  56. A. Sheth and J. Larson, “Federated database systems for managing distributed, heterogeneous, and autonomous databases,” ACM Computing Surveys 22(3), 1990, 183–236.

    Google Scholar 

  57. SOAP, “Simple object access protocol (SOAP) 1.2,” W3C Recommendation, http://www.w3.org/ TR/soap12-part0, 2003.

  58. M. Stonebraker, “The design and implementation of distributed INGRES,” in The INGRES Papers: Anatomy of a Relational Database System, Addison-Wesley, Reading, MA, USA, 1985.

    Google Scholar 

  59. M. Stonebraker, E. Anderson, E. Hanson, and B. Rubenstein, “QUEL as a data type,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Boston, MA, 1984, pp. 208–214.

  60. M. Stonebraker, P. Aoki, W. Litwin, A. Pfeffer, A. Sah, J. Sidell, C. Staelin, and A. Yu, “Mariposa: A Wide-area distributed database system,” The VLDB Journal 5(1), 1996, 48–63.

    Google Scholar 

  61. SunONE, “Sun Open Net Environment (Sun ONE),” http://www.sun.com/sunone

  62. A. Tomasic, L. Raschid, and P. Valduriez, “Scaling heterogeneous databases and the design of DISCO,” in Proceedings of the International Conference on Distributed Computing Systems, Hong Kong, 1996, pp. 449–457.

  63. TPC, “TPC Benchmark D (decision support),” Standard Specification 2.1, Transaction Processing Performance Council (TPC), http://www.tpc.org, 1999.

  64. WebSphere, “IBM WebSphere,” http://www.ibm.com/websphere

  65. G. Wiederhold, “Intelligent integration of information,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Washington, DC, USA, 1993, pp. 434–437.

  66. C. Wiesner, P. Winklhofer, and A. Kemper, “Building dynamic market places using HyperQueries,” in Proceedings of the International Conference on Extending Database Technology (EDBT), Prague, Czech Republic, 2002, pp. 749–752.

  67. R. Williams, D. Daniels, L. Haas, G. Lapis, B. Lindsay, P. Ng, R. Obermarck, P. Selinger, A. Walker, P. Wilms, and R. Yost, “R *: An overview of the architecture,” IBM Research, San Jose, CA, RJ3325, 1981. Reprinted in M. Stonebraker <nt>(ed.)</nt>, Readings in Database Systems, Morgan Kaufmann, San Mateo, CA, 1994, pp. 515–536.

    Google Scholar 

  68. L. Xyleme, “A dynamic warehouse for XML data of the Web,” IEEE Data Engineering Bulletin 24(2), 2001, 40–47.

    Google Scholar 

  69. J. Yang and M. P. Papazoglou, “Interoperation support for electronic commerce,” Communications of the ACM 43(6), 2000, 39–47.

    Google Scholar 

  70. V. Zadorozhny, L. Raschid, M. E. Vidal, T. Urhan, and L. Bright, “Efficient evaluation of queries in a mediator for WebSources,” in Proceedings of the ACM SIGMOD Conference on Management of Data, Madison, WI, USA, 2002, pp. 85–96.

Download references

Author information

Authors and Affiliations

  1. Institut für Informatik, Technische Universität München, 85748, Garching bei München, Germany

    Alfons Kemper

  2. Lehrstuhl für Informatik, Universität Passau, 94030, Passau, Germany

    Christian Wiesner

Authors
  1. Alfons Kemper
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Wiesner
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kemper, A., Wiesner, C. Building Scalable Electronic Market Places Using HyperQuery-Based Distributed Query Processing. World Wide Web 8, 27–60 (2005). https://doi.org/10.1023/B:WWWJ.0000047379.18584.31

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:WWWJ.0000047379.18584.31

Search

Navigation