Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Modeling Data Stream Intensity in Distributed Stream Processing System

  • Conference paper
Computer Networks (CN 2013)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 370))

Included in the following conference series:

  • 1558 Accesses

Abstract

In recent years energy market has changed. Consumers in many countries are free to buy energy from any of the available providers. This requires continuous reading from a huge number of energy meters to evaluate the amount of energy being bought from a particular provider. In this paper we present a fault-tolerant distributed stream processing system for continuous meter readings. The main goal of the system is to store the readings in a stream data warehouse for further analysis. We focus on modeling of the data stream intensity in order to estimate the size of buffers in a network of components composing the system. We present both the mathematical model of the intensity and the simulation results to prove the correctness of the theoretical analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Wrembel, R.: On Handling the Evolution of External Data Sources in a Data Warehouse Architecture: Integrations of Data Warehousing. In: Taniar, D., Chen, L. (eds.) Integrations of Data Warehousing, Data Mining and Database Technologies, pp. 106–147 (2011)

    Google Scholar 

  2. Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., Widom, J.: Stream: The stanford stream data manager. IEEE Data Eng. Bull. 26(1), 19–26 (2003)

    Google Scholar 

  3. Gorawski, M., Chrószcz, A.: Synchronization Modeling in Stream Processing. In: Morzy, T., Härder, T., Wrembel, R. (eds.) Advances in Databases and Information Systems. AISC, vol. 186, pp. 91–102. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  4. Gorawski, M., Chroszcz, A.: Optimization of operator partitions in stream data warehouse. In: Song, L.-Y., Cuzzocrea, A., Davis, K.C. (eds.) DOLAP 2011, pp. 61–66. ACM (2011)

    Google Scholar 

  5. Gorawski, M., Malczok, R.: Indexing Spatial Objects in Stream Data Warehouse. In: Nguyen, N.T., Katarzyniak, R., Chen, S.-M. (eds.) Advances in Intelligent Information and Database Systems. SCI, vol. 283, pp. 53–65. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  6. Gorawski, M., Malczok, R.: Answering Range-Aggregate Queries over Objects Generating Data Streams. In: Kitagawa, H., Ishikawa, Y., Li, Q., Watanabe, C. (eds.) DASFAA 2010. LNCS, vol. 5982, pp. 436–439. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  7. Gorawski, M.: Multiversion Spatio-temporal Telemetric Data Warehouse. In: Grundspenkis, J., Kirikova, M., Manolopoulos, Y., Novickis, L. (eds.) ADBIS 2009. LNCS, vol. 5968, pp. 63–70. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Kwiecień, A., Opielka, K.: Industrial Networks in Explosive Atmospheres. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 367–378. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  9. Kwiecień, A., Stój, J.: Genius Network Communication Process Registration and Analysis. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 314–321. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  10. Balazinska, M., Balakrishnan, H., Madden, S., Stonebraker, M.: Fault-Tolerance in the Borealis Distributed Stream Processing System. In: ACM SIGMOD Conf., Baltimore, MD (2005)

    Google Scholar 

  11. Madden, S., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data: ICDE. In: Proceedings of the 18th International Conference on Data Engineering, pp. 555–566. IEEE Computer Society (2002)

    Google Scholar 

  12. Gorawski, M., Malczok, R.: Distributed spatial data warehouse indexed with virtual memory aggregation tree. In: Sander, J., Nascimento, M.A. (eds.) STDBM, pp. 25–32 (2004)

    Google Scholar 

  13. Gorawski, M., Bańkowski, S., Gorawski, M.: Selection of Structures with Grid Optimization in Multiagent Data Warehouse. In: Fyfe, C., Tino, P., Charles, D., Garcia-Osorio, C., Yin, H. (eds.) IDEAL 2010. LNCS, vol. 6283, pp. 292–299. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  14. Gorawski, M., Marks, P.: High efficiency of hybrid resumption in distributed data warehouses. In: DEXA Workshops, pp. 323–327. IEEE Computer Society (2005)

    Google Scholar 

  15. Gorawski, M., Marks, P.: Checkpoint-based resumption in data warehouses. In: Socha, K. (ed.) IFIP International Federation for Information Processing. Software Engineering Techniques: Design for Quality, vol. 227, pp. 313–323. Springer, Boston (2006)

    Google Scholar 

  16. Labio, W., Wiener, J.L., Garcia-Molina, H., Gorelik, V.: Efficient resumption of interrupted warehouse loads. In: Chen, W., Naughton, J.F., Bernstein, P.A. (eds.) SIGMOD Conference, pp. 46–57. ACM (2000)

    Google Scholar 

  17. Gorawski, M., Marks, P.: Fault-tolerant distributed stream processing system. In: DEXA Workshops, pp. 395–399. IEEE Computer Society (2006)

    Google Scholar 

  18. Gorawski, M., Marks, P.: Towards reliability and fault-tolerance of distributed stream processing system. In: DepCoS-RELCOMEX, pp. 246–253. IEEE Computer Society (2007)

    Google Scholar 

  19. Gorawski, M., Marks, P.: Distributed stream processing analysis in high availability context. In: ARES 2007: Proceedings of the The Second International Conference on Availability, Reliability and Security, pp. 61–68. IEEE Computer Society, Washington, DC (2007)

    Chapter  Google Scholar 

  20. Gorawski, M., Marks, P.: Towards automated analysis of connections network in distributed stream processing system. In: Haritsa, J.R., Kotagiri, R., Pudi, V. (eds.) DASFAA 2008. LNCS, vol. 4947, pp. 670–677. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  21. Gorawski, M., Marks, P., Gorawski, M.: Collecting data streams from a distributed radio-based measurement system. In: Haritsa, J.R., Kotagiri, R., Pudi, V. (eds.) DASFAA 2008. LNCS, vol. 4947, pp. 702–705. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computer Science, Silesian University of Technology, Akademicka 16, 44-100, Gliwice, Poland

    Marcin Gorawski & Pawel Marks

  2. Institute of Computer Science, Wroclaw University of Technology, Wybrzeze Wyspianskiego 27, 50-370, Wroclaw, Poland

    Marcin Gorawski

  3. Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Baltycka 5, 44-100, Gliwice, Poland

    Michal Gorawski

Authors
  1. Marcin Gorawski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pawel Marks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michal Gorawski
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Informatics, Silesian University of Technology, ul. Akademicka 16, 41-100, Gliwice, Poland

    Andrzej Kwiecień

  2. Institute of Informatics, Silesian University of Technology, ul. Akademicka 16, 44-100, Gliwice, Poland

    Piotr Gaj  & Piotr Stera  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gorawski, M., Marks, P., Gorawski, M. (2013). Modeling Data Stream Intensity in Distributed Stream Processing System. In: Kwiecień, A., Gaj, P., Stera, P. (eds) Computer Networks. CN 2013. Communications in Computer and Information Science, vol 370. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38865-1_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38865-1_38

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38864-4

  • Online ISBN: 978-3-642-38865-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation