Skip to main content
Springer Nature Link
Log in

Stream-Based Real World Information Integration Framework

  • Chapter
Wireless Sensor Network Technologies for the Information Explosion Era

Part of the book series: Studies in Computational Intelligence ((SCI,volume 278))

Abstract

For real world oriented applications to easily use sensor data obtained frommultiple wireless sensor networks, a data management infrastructure is mandatory. The infrastructure design should be based on the philosophy of a novel framework beyond the relational data management for two reasons: First is the freshness of data. To keep sensor data fresh, an infrastructure should process data efficiently; this means conventional time consuming transaction processing methodology is inappropriate. Second is the diversity of functions. The primary purpose of sensor data applications is to detect events; unfortunately, relational operators contribute little toward this purpose. This chapter presents a framework that directly supports efficient processing and a variety of advanced functions. Stream processing is the key concept of the framework. Regarding the efficiency requirement, we present a multiple query optimization technique for query processing over data streams; we also present an efficient data archiving technique. To meet the functions requirement, we present several techniques on event detection, which include complex event processing, probabilistic reasoning, and continuous media integration.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Abadi, D.J., Ahmad, Y., Balazinska, M., Cetintemel, U., Cherniack, M., Hwang, J.H., Lindner, W., Maskey, A., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S.B.: The design of the borealis stream processing engine. In: Proc. CIDR (2005)

    Google Scholar 

  2. Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment. In: Proc. VLDB, pp. 1199–1202 (2006)

    Google Scholar 

  3. Agrawal, J., Diao, Y., Gyllstrom, D., Immerman, N.: Efficient pattern matching over event streams. In: Proc. ACM SIGMOD (2008)

    Google Scholar 

  4. Agrawal, P., Benjelloun, O., Sarma, A.D., Hayworth, C., Nabar, S., Sugihara, T., Widom, J.: Trio: A system for data, uncertainty, and lineage. In: Proc. of VLDB, pp. 1151–1154 (2006)

    Google Scholar 

  5. Arasu, A., Babu, S., Widom, J.: The cql continuous query language: Semantic foundations and query execution. VLDB Journal 15(2) (2006)

    Google Scholar 

  6. Arora, A., Ramnath, R., Ertin, E., Sinha, P., Bapat, S., Naik, V., Kulathumani, V., Zhang, H., Cao, H., Sridharan, M., Kumar, S., Seddon, N., Anderson, C., Herman, T., Trivedi, N., Zhang, C., Shah, R., Kulkarni, S., Aramugam, M., Wang, L.: Exscal: Elements of an extreme scale wireless sensor network. In: Proc. of IEEE RTCSA, pp. 102–108 (2005)

    Google Scholar 

  7. Avnur, R., Hellerstein, J.M.: Eddies: Continuously adaptive query processing. In: Proc. ACM SIGMOD, pp. 261–272 (2000)

    Google Scholar 

  8. Ayad, A.M., Naughton, J.F.: Static optimization of conjunctive queries with sliding windows over infinite streams. In: Proc. ACM SIGMOD, pp. 419–430 (2004)

    Google Scholar 

  9. Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring streams – a new class of data management applictions. In: Proc. VLDB, pp. 215–226 (2002)

    Google Scholar 

  10. Chandrasekaran, S., Franklin, M.J.: Streaming queries over streaming data. In: Proc. VLDB, pp. 203–214 (2002)

    Google Scholar 

  11. Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. In: Proc. OSDI (2006)

    Google Scholar 

  12. Chen, J., DeWitt, D., Naughton, J.: Design and evaluation of alternative selection placement strategies in optimizing continuous queries. In: Proc. IEEE ICDE, pp. 345–356 (2002)

    Google Scholar 

  13. Cooper, B.F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.A., Puz, N., Weaver, D., Yerneni, R.: Pnuts: Yahoo! ’s hosted data serving platform. In: Proc. VLDB (2008)

    Google Scholar 

  14. Demers, A., Gehrke, J., Hong, M., Riedewald, M., White, W.: Towards expressive publish/subscribe systems. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., Böhm, K., Kemper, A., Grust, T., Böhm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 627–644. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  15. Deshpande, A., Madden, S.: Mauvedb: Supporting model-based user views in database systems. In: Proc. ACM SIGMOD (2006)

    Google Scholar 

  16. Gedik, B., Liu, L.: Peercq: A decentralized and self-configuring peer-to-peer informaiton monitoring system. In: Proc. ICDCS, pp. 490–499 (2003)

    Google Scholar 

  17. Kadota, M., Aida, H., Nakazawa, J., Tokuda, H.: D-jenga: A parallel distributed bayesian inference mechanism on wireless sensor nodes. In: Proc. of International Conference on Networked Sensing Systems (2006)

    Google Scholar 

  18. Kanzaki, A., Hara, T., Ishi, Y., Wakamiya, N., Shimojo, S.: X-sensor: a sensor network testbed integrating multi-networks. In: Proc. of Int’l Workshop on Data Management for Information Explosion in Wireless Networks (DMIEW 2009), pp. 1082–1087 (2009)

    Google Scholar 

  19. Li, J., Tufte, K., Shkapenyuk, V., Papadimos, V., Johnson, T., Maier, D.: Out-of-order processing: A new architecture for high-performance stream systems, pp. 274–288 (2008)

    Google Scholar 

  20. Liu, L., Pu, C., Tang, W.: Continual queries for internet scale event-driven information delivery. IEEE TKDE 11(4), 610–628 (1999)

    Google Scholar 

  21. Madden, S., Franklin, M., Hellerstein, J., Hong, W.: The design of an acquisitional query processor for sensor networks. In: Proc. ACM SIGMOD, pp. 491–502 (2003)

    Google Scholar 

  22. Madden, S., Shah, M., Hellerstein, J., Raman, V.: Continuously adaptive continuous queries over streams. In: Proc. ACM SIGMOD, pp. 49–60 (2002)

    Google Scholar 

  23. Maekawa, T., Yanagisawa, Y., Sakurai, Y., Kishino, Y., Kamei, K., Okadome, T.: Web searching for daily living. In: Proc. ACM SIGIR, pp. 27–34 (2009)

    Google Scholar 

  24. Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G.S., Olston, C., Rosenstein, J., Varma, R.: Query processing, resource management, and approximation in a data stream management system. In: Proc. CIDR (2003)

    Google Scholar 

  25. Ohki, K., Watanabe, Y., Kitagawa, H.: Evaluation of a framework for dynamic source selection in stream processing. In: Proc. International Workshop on Data Management for Information Explosion in Wireless Networks, DMIEW 2009 (2009)

    Google Scholar 

  26. Roy, P., Seshadri, S., Sudarshan, S., Bhobe, S.: Efficient and extensible algorithms for multi query optimization. In: Proc. ACM SIGMOD, pp. 249–260 (2000)

    Google Scholar 

  27. Salton, G.: Automatic Information Organization and Retrieval. McGraw-Hill Book Company, New York (1968)

    Google Scholar 

  28. Sato, R., Kawashima, H., Kitagawa, H.: The integration of data streams with probabilities and a relational database using bayesian networks. In: Proc. of IEEE International Workshop on Sensor Network Technologies for Information Explosion Era, SeNTIE (2008)

    Google Scholar 

  29. Sellis, T.: Multiple-query optimization. ACM TODS 13(1), 23–52 (1988)

    Article  Google Scholar 

  30. Shen, Z., Kawashima, H., Kitagawa, H.: Efficient probabilistic event stream processing with lineage and kleene-plus. International Journal of Communication Networks and Distributed Systems 2(4), 355–374 (2009)

    Article  Google Scholar 

  31. Stonebraker, M., Çetintemel, U.: One size fits all: An idea whose time has come and gone. In: Proc. IEEE ICDE, pp. 2–11 (2005)

    Google Scholar 

  32. StreamSpinner Team: http://www.streamspinner.org

  33. Terry, D., Goldberg, D., Nichols, D.: Continuous queries over append-only databases. In: Proc. ACM SIGMOD, pp. 321–330 (1992)

    Google Scholar 

  34. Tran, T., Sutton, C., Cocci, R., Nie, Y., Diao, Y., Shenoy, P.: Probabilistic inference over rfid streams in mobile environments. In: Proc. IEEE ICDE (2009)

    Google Scholar 

  35. Watanabe, Y., Akiyama, R., Ohki, K., Kitagawa, H.: A video stream management system for heterogeneous information integration environments. In: Proc. of 2nd International Conference on Ubiquitous Information Management and Communication, ICUIMC 2008 (2008)

    Google Scholar 

  36. Watanabe, Y., Kitagawa, H.: A multiple continuous query optimization method based on query execution pattern analysis. In: Lee, Y., Li, J., Whang, K.-Y., Lee, D. (eds.) DASFAA 2004. LNCS, vol. 2973, pp. 443–456. Springer, Heidelberg (2004)

    Google Scholar 

  37. Watanabe, Y., Kitagawa, H.: Query result caching for multiple event-driven continuous queries. Information Systems (2009) (to appear)

    Google Scholar 

  38. Watanabe, Y., Yamada, S., Kitagawa, H., Amagasa, T.: Integrating a stream processing engine and databases for persistent streaming data management. In: Wagner, R., Revell, N., Pernul, G. (eds.) DEXA 2007. LNCS, vol. 4653, pp. 414–423. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  39. Wyss, C.M., Wyss, F.I.: Extending relational query optimization to dynamic schemas for information integration in multidatabases. In: Proc. ACM SIGMOD, pp. 473–484 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Systems and Information Engineering and Center for Computational Sciences, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8573, Japan

    Hiroyuki Kitagawa, Hideyuki Kawashima & Toshiyuki Amagasa

  2. Global Scientific Information and Computing Center, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro-ku, Tokyo, 152-8550, Japan

    Yousuke Watanabe

Authors
  1. Hiroyuki Kitagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yousuke Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hideyuki Kawashima
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Toshiyuki Amagasa
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Multimedia Engineering Graduate School of Information Science and Technology, Osaka University, 1-5 Yamadaoka Suita, 565-0871, Osaka, Japan

    Takahiro Hara

  2. Department of Information Science and Telecommunications School of Information Sciences, University of Pittsburgh, 135 North BellefieldAve, PA 15260, Pittsburgh, USA

    Vladimir I. Zadorozhny

  3. Institute for Program Structures und Data Organization, Karlsruhe Institute of Technology, Geb. 50.34 Am Fasanengarten 5, D-76131, Karlsruhe, Germany

    Erik Buchmann

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kitagawa, H., Watanabe, Y., Kawashima, H., Amagasa, T. (2010). Stream-Based Real World Information Integration Framework. In: Hara, T., Zadorozhny, V.I., Buchmann, E. (eds) Wireless Sensor Network Technologies for the Information Explosion Era. Studies in Computational Intelligence, vol 278. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13965-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-13965-9_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-13964-2

  • Online ISBN: 978-3-642-13965-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics