Skip to main content
SpringerLink
Log in

Informix under CONTROL: Online Query Processing

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

The goal of the CONTROL project at Berkeley is to develop systems for interactive analysis of large data sets. We focus on systems that provide users with iteratively refining answers to requests and online control of processing, thereby tightening the loop in the data analysis process. This paper presents the database-centric subproject of CONTROL: a complete online query processing facility, implemented in a commercial Object-Relational DBMS from Informix. We describe the algorithms at the core of the system, and detail the end-to-end issues required to bring the algorithms together and deliver a complete system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agrawal, R. 1997. Personal communication.

  • Agrawal, R. and Srikant, R. 1994. Fast algorithms for mining association rules. In Proc. 20th International Conference on Very Large Data Bases, Santiago de Chile, September 1994.

  • Aiken, A., Chen, J., Stonebraker, M., and Woodruff, A. 1996.Tioga-2:Adirect-manipulation database visualization environment. In Proc. 12th IEEE International Conference on Data Engineering, New Orleans, February 1996.

  • Antoshenkov, G. 1992. Random sampling from pseudo-ranked B+ trees. In Proc. 18th International Conference on Very Large Data Bases, Vancouver, August 1992.

  • Antoshenkov, G. and Ziauddin, M. 1996. Query processing and optimization in Oracle Rdb. VLDB Journal, 5(4):229–237.

    Article  Google Scholar 

  • Aoki, P.M. 1998. Generalizing “search” in generalized search trees. In IEEE International Conference on Data Engineering, Orlando, February 1998.

  • Astrahan, M., Blasgen, M., Chamberlin, D., Eswaran, K., Gray, J., Griffiths, P., King, W., Lorie, R., McJones, P., Mehl, J., Putzolu, G., Traiger, I., Wade, B., and Watson, V. 1976. System R: Relational approach to database management. ACM Transactions on Database Systems, 1(2):97–137.

    Article  Google Scholar 

  • Avnur, R. and Hellerstein, J.M. 2000. Eddies: Continuously adaptive query processing. In Proc. ACM-SIGMOD International Conference on Management of Data, Dallas, May 2000.

  • Bayardo Jr., R.J. and Miranker, D.P. 1996. Processing queries for first-few answers. In Fifth Intl. Conf. Information and Knowledge Management, Rockville, MD.

  • Carey, M.J. and Kossmann, D. 1997. On saying “Enough Already!” in SQL. In Proc. ACM-SIGMOD International Conference on Management of Data, Tucson, May 1997.

  • Carey, M.J. and Kossmann, D. 1998. Reducing the braking distance of an SQL query engine. In Proc. 24th International Conference on Very Large Data Bases, New York City.

  • Chaudhuri, S. and Gravano, L. 1996. Optimizing queries over multimedia repositories. In Proc. ACM-SIGMOD International Conference on Management of Data, Montreal, June 1996.

  • Chaudhuri, S. and Gravano, L. 1999. Evaluating top-k selection queries. In Proc. International Conference on Very Large Data Bases, Edinburgh.

  • Chaudhuri, S. and Narasayya, V. 1998. AutoAdmin “What-If” index analysis utility. In Proc. ACM-SIGMOD International Conference on Management of Data, Seattle, June 1998.

  • Chaudhuri, S. and Shim, K. 1996. Optimization of queries with user-defined predicates. In Proc. 24th International Conference on Very Large Data Bases, Bombay (Mumbai), September 1996.

  • Cherniack, M. 1998. Building query optimizers with combinators. PhD Thesis, Brown University.

  • DeWitt, D.J., Katz, R.H., Olken, Frank, Shapiro, L.D., Stonebraker, R.M., and Wood, D. 1984. Implementation techniques for main memory database systems. In Proc. ACM-SIGMOD International Conference on Management of Data, Boston, June 1984.

  • Donjerkovic, D. and Ramakrishnan, R. 1999. Probabilistic optimization of Top N queries. In Proc. International Conference on Very Large Data Bases, Edinburgh.

  • Fagin, R. 1998. Fuzzy queries in multimedia database systems. In Proc. ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, Seattle, June 1998.

  • Fayyad, U., Piatetsky-Shapiro, G., and Smyth, P. 1996. The kdd process for extracting useful knowledge from volumes of data. Communications of the ACM, 39(11).

  • Fushimi, S., Kitsuregawa, M., and Tanaka, H. 1986. An overview of the system software of a parallel relational database machine GRACE. In Proc. 24th International Conference on Very Large Data Bases, Kyoto, August 1986.

  • Gibbons, P.B. and Matias, Y. 1998. New sampling-based summary statistics for improving approximate query answers. In Proc. ACM-SIGMOD International Conference on Management of Data, Seattle.

  • Gibbons, P.B., Poosala, V., Acharya, S., Bartal, Y., Matias, Y., Muthukrishnan, S., Ramaswamy, S., and Suel, T. 1998. Aqua: System and techniques for approximate query answering. Technical Report, Bell Laboratories.

  • Graefe, G. 1993. Query evaluation techniques for large databases. ACM Computing Surveys, 25(2):73–170.

    Article  Google Scholar 

  • Gray, J. and Graefe, G. 1997. The five-minute rule ten years later, and other computer storage rules of thumb. SIGMOD Record, 26(4).

  • Haas, P.J. 1996. Hoeffding inequalities for join-selectivity estimation and online aggregation. IBM Research Report RJ 10040, IBM Almaden Research Center.

  • Haas, P.J. 1997. Large-sample and deterministic confidence intervals for online aggregation. In Proc. 9th International Conference on Scientific and Statistical Database Management, Olympia, WA, August 1997.

  • Haas, P.J. and Hellerstein, J.M. 1999. Ripple algorithms for online aggregation. In Proc. ACM-SIGMOD International Conference on Management of Data, Philadelphia, May 1999.

  • Haas, P.J., Naughton, J.F., Seshadri, S., and Swami, A.N. 1996. Selectivity and cost estimation for joins based on random sampling. Journal of Computer System Science, 52:550–569.

    Article  MathSciNet  Google Scholar 

  • Harinarayan, V., Rajaraman, A., and Ullman, J.D. 1996. Implementing data cubes efficiently. In Proc. ACMSIGMOD International Conference on Management of Data, Montreal, June 1996.

  • Hellerstein, J.M. 1997a. The case for online aggregation. Computer Science Technical Report CSD-97-958, University of California, Berkeley.

    Google Scholar 

  • Hellerstein, J.M. 1997b. Online processing redux. IEEE Data Engineering Bulletin, 20(3).

  • Hellerstein, J.M. 1998a. Looking forward to interactive queries. Database Programming and Design, 11(8):28–33.

    Google Scholar 

  • Hellerstein, J.M. 1998b. Optimization techniques for queries with expensive predicates. ACM Transactions on Database Systems, 23(2).

  • Hellerstein, J.M., Avnur, R., Chou, A., Hidber, C., Olston, C., Raman, V., and Roth, T. 1999. Interactive Data Analysis with CONTROL. IEEE Computer 32(9):51–59.

    Google Scholar 

  • Hellerstein, J.M., Haas, P.J., and Wang, H.J. 1997. Online aggregation. In Proc. ACM-SIGMOD International Conference on Management of Data, Tucson, May 1997.

  • Hellerstein, J.M. and Naughton, J.F. 1996. Query execution techniques for caching expensive methods. In Proc. ACM-SIGMOD International Conference on Management of Data, Montreal, June 1996.

  • Hidber, C. 1997. Online association rule mining. In Proc. ACM-SIGMOD International Conference on Management of Data, Tucson, May 1997.

  • Hoeffding, W. 1963. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58.

  • Hou, W.C., Ozsoyoglu, G., and Taneja, B.K. 1988. Statistical estimators for relational algebra expressions. In Proc. 7th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, Austin, March 1998.

  • Hou, W.C., Ozsoyoglu, G., and Taneja, B.K. 1989. Processing aggregate relational queries with hard time constraints. In Proc. ACM-SIGMOD International Conference on Management of Data, Portland, May-June 1989.

  • Hyperion Essbase OLAP Server, 1998. URL http://www.hyperion.com/essbaseolap.cfm.

  • Illustra Information Technologies, Inc. 1994. Illustra User's Guide, Illustra Server Release 2.1.

  • Informix Corp. 1998a. Sampling: The latest breakthrough in decision support technology. Informix White Paper 000-21681-70.

  • Informix Corp. 1998b. C-ISAM Version 7.24 for the UNIX Operating System.

  • Informix Corp. 1998c. Informix Dynamic Server with Universal Data Option 9.1x.

  • Knuth, D.E. 1973. The Art of Computer Programming: Vol. 3, Sorting and Searching. Addison-Wesley.

  • Leung, T.Y.C., Pirahesh, H., Seshadri, P., and Hellerstein, J.M. 1998. Query rewrite optimization rules in IBM DB/2 universal database. In Readings in Database Systems, 3rd ed., M. Stonebraker and J.M. Hellerstein (Eds.). San Francisco: Morgan-Kaufmann.

    Google Scholar 

  • Lipton, R.J., Naughton, J.F., Schneider, D.A., and Seshadri, S. 1993. Efficient sampling strategies for relational database operations. Theoretical Computer Science, 116:195–226.

    Article  MathSciNet  Google Scholar 

  • Livny, M., Ramakrishnan, R., Beyer, K.S., Chen, G., Donjerkovic, D., Lawande, S., and Myllymaki, J. 1997. DEVise: Integrated querying and visualization of large datasets. In Proc. ACM-SIGMOD International Conference on Management of Data, Tucson, May 1997.

  • Lynch, C. and Stonebraker, M. 1988. Extended user-defined indexing with application to textual databases. In Proc. 14th International Conference on Very Large Data Bases, Los Angeles, August-Septeber 1998.

  • Maier, D. and Stein, J. 1986. Indexing in an object-oriented DBMS. In Proc. 1st Workshop on Object-Oriented Database Systems, Asilomar, September 1986.

  • Morgenstein, J.P. 1980. Computer based management information systems embodying answer accuracy as a user parameter. PhD Thesis, U.C. Berkeley.

  • O'day, V. and Jeffries, R. 1993. Orienteering in an information landscape: How information seekers get from here to there. In INTERCHI.

  • Ohno, P. 1998. Visionary. Informix Magazine.

  • Olken, F. 1993. Random sampling from databases. PhD Thesis, University of California, Berkeley.

    Google Scholar 

  • Papadopoulos, G. Chief Technology Officer. 1997. Sun Microsystems. Untitled talk. Berkeley NOWRetreat, July 1997.

  • Perlin, K. and Fox, D. 1993. Pad: An alternative approach to the computer interface. In Proc. ACM SIGGRAPH, Anaheim, pp. 57–64.

  • Pilot Software 1998. Announces release of PDSS 6.0. URL http://www.pilotsw.com/about/pressrel/pr72998.htm.

  • Raman, V., Chou, A., and Hellerstein, J.M. 1999a. Scalable spreadsheets for interactive data analysis. In DMKD Workshop.

  • Raman, V., Raman, B., and Hellerstein, J.M. 1999b. Online dynamic reordering for interactive data processing. In Proc. International Conference on Very Large Data Bases, Edinburgh.

  • Rao, J. and Ross, K.A. 1998. Reusing invariants: A new strategy for correlated queries. In Proc. ACM-SIGMOD International Conference on Management of Data, Seattle, June 1998.

  • Red Brick Systems, Inc. 1998. Red brick warehouse. URL http://www.redbrick.com/products/rbw/rbw.html.

  • Seshadri, P. and Swami, A. 1995. Generalized partial indexes. In Proc. 11th IEEE International Conference on Data Engineering, Taipei, March 1995.

  • Shneiderman, B. 1982. The future of interactive systems and the emergence of direct manipulation. Behavior and Information Technology, 1(3):237–256.

    Google Scholar 

  • Shukla, A., Deshpande, P., and Naughton, J.F. 1998. Materialized view selection for multidimensional datasets. In Proc. 24th International Conference on Very Large Data Bases, New York City.

  • Silberschatz, A., Read, R.L., and Fussell, D.S. 1992. A multi-resolution relational data model. In Proc. 18th International Conference on Very Large Data Bases, Vancouver, August 1992.

  • QL 1998. Server 7.0 OLAP services. URL http://www.microsoft.com/backoffice/sql/70/whpprs/olapoverview.htm.

  • Stonebraker, M. 1989. The case for partial indexes. SIGMOD Record, 18(4):4–11.

    Article  Google Scholar 

  • Stonebraker, M. and Kemnitz, G. 1991. The POSTGRES Next-Generation database management system. Communications of the ACM, 34(10):78–92.

    Article  Google Scholar 

  • Tan, K., Goh, C.H., and Ooi, B.C. 1999. Online feedback for nested aggregate queries with multi-threading. In Prov. International Conference on Very Large Data Bases, Edinburgh.

  • Transaction Processing Council. TPC-D Rev. 1.2.3 Benchmark Specification.URLhttp://www.tpc.org/dspec.html.

  • Vrbsky, S.V. and Liu, J.W.S. 1993. APPROXIMATE-A query processor that produces monotonically improving approximate answers. IEEE Transactions on Knowledge and Data Engineering, 5(6):1056–1068.

    Article  Google Scholar 

  • Waldspurger, C.A. and Weihl, W.E. 1995. Lottery scheduling: Flexible proportional-share resource management. In First Symposium on Operating Systems Design and Implementation (OSDI).

  • Walter, T., Chief Technical Officer. 1998. NCR parallel systems. Complex queries. NSF Database Systems Industrial/Academic Workshop, October 1998.

  • Wilschut, A.N. and Apers, P.M.G. 1991. Dataflow query execution in a parallel main-memory environment. In Proc. First Intl. Conf. Parallel and Distributed Info. Sys. (PDIS), pages 68-77, Miami Beach, December 1991.

  • Winter, R. and Auerbach, K. 1998. The big time: 1998 winter VLDB survey. Database Programming and Design.

  • Zilberstein, S. and Russell, S.J. 1996. Optimal composition of real-time systems. Artificial Intelligence, 82(1/2):181–213.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Division, U.C., Berkeley, USA

    Joseph M. Hellerstein, Ron Avnur & Vijayshankar Raman

Authors
  1. Joseph M. Hellerstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ron Avnur
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vijayshankar Raman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hellerstein, J.M., Avnur, R. & Raman, V. Informix under CONTROL: Online Query Processing. Data Mining and Knowledge Discovery 4, 281–314 (2000). https://doi.org/10.1023/A:1009835310546

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1009835310546

Search

Navigation