Skip to main content
SpringerLink
Log in

Reinforcement learning-based prediction approach for distributed Dynamic Data-Driven Application Systems

  • Published:
Information Technology and Management Aims and scope Submit manuscript

Abstract

With the current advances in cloud and distributed system technology, data have become ubiquitous and their dynamics has increased. It is an extreme challenge to find the interdependencies among distributed data in order to dynamically manage and predict the trend within large amounts of data sources. This paper proposes a new distributed dynamic data-driven model and strategy to direct and evaluate the interlinked data sets in Dynamic Data-Driven Application Systems (DDDAS). The underlying technique involves the introduction of a reinforcement Q-Learning approach which includes search strategies to determine how to drill and drive a series of highly dependent data in order to enhance prediction accuracy and efficiency. It can tackle dynamic data issues in a real-time, dynamic and resource-bounded environment. The proposed framework is a comprehensive skeleton for modeling complex, flexible and dynamic tasks in a distributed environment for solving DDDAS problems. In simulation, the new model utilizes individual sensors, distributed databases and predictors in Dynamic Data Stream Nodes with multiple dimensional variables which can be instantiated to explore the search space, thereby improving the search convergence. This study shows the effectiveness and applicability of using the technique in the analysis of typhoon rainfall data. The result shows that the proposed approach performed better than traditional linear regression approaches, reducing the error rate by 36.34 %.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Liu B, Cao S, He W (2011) Distributed data mining for e-business. Inf Technol Manag 12:67–79

  2. Aggarwal CC, Xie Y, Yu PS (2011) On dynamic data-driven selection of sensor streams. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, CA, USA, pp 1226–1234

  3. He W, Da XL (2014) Integration of distributed enterprise applications: a survey. IEEE Trans Ind Inf 10:35–42

    Article  Google Scholar 

  4. Darema F (2005) Dynamic Data Driven Applications Systems: new capabilities for application simulations and measurements. In: Computational science—ICCS 2005 (lecture notes in computer science), vol 3515, pp 661–712

  5. Darema F (2004) Dynamic Data Driven Applications Systems: a new paradigm for application simulations and measurements. In: Computational science—ICCS 2004 (lecture notes in computer science), vol 3038, pp 662–669

  6. NSF. DDDAS 2006 workshop report (online). http://www.dddas.org/nsf-workshop-2006/wkshp_report.pdf

  7. Lin S-Y, Chao K-M, Lo C-C, Godwin N (2013) Distributed dynamic data driven prediction based on reinforcement learning approach. In: Proceedings of the 28th annual ACM symposium on applied computing (SAC '13), New York, USA, pp 779–784

  8. Mahadevan S, Kaelbling LP (1996) The NSF workshop on reinforcement learning: summary and observations. AI Mag 17:1–16

  9. Bar-Hillel A, Di-Nur A, Ein-Dor L, Gilad-Bachrach R, Ittach Y (2007) Workstation capacity tuning using reinforcement learning. In: Proceedings of the 2007 ACM/IEEE conference on supercomputing, Reno, Nevada, pp 1–11

  10. Huang Z, van der Aalst WMP, Lu X, Duan H (2011) Reinforcement learning based resource allocation in business process management. Data Knowl Eng 70:127–145

    Article  Google Scholar 

  11. NSF. DDDAS 2000 workshop report (online). http://www.nsf.gov/cise/cns/dddas/dd_das_work_shop_rprt.pdf

  12. Darema F (2007) Introduction to the ICCS 2007 workshop on dynamic data driven applications systems. In: Computational science—ICCS 2007 (lecture notes in computer science), vol 4487, pp 955–962

  13. Ouyang Y, Zhang JE, Luo SM (2007) Dynamic data driven application system: recent development and future perspective. Ecol Model 204:1–8

    Article  Google Scholar 

  14. Ouyang Y, Luo SM, Cui LH, Wang Q, Zhang JE (2011) Estimation of real-time N load in surface water using Dynamic Data-Driven Application System. Ecol Eng 37:616–621

    Article  Google Scholar 

  15. Patrikalakis N, McCarthy J, Robinson A, Schmidt H, Evangelinos C, Haley P, Lalis S, Lermustaux P, Tian R, Leslie W (2004) Towards a dynamic data driven system for rapid adaptive interdisciplinary ocean forecasting, Invited paper in Dynamic Data-Driven Application Systems. Kluwer, Amsterdam

    Google Scholar 

  16. Douglas CC, Lodder RA, Beezley JD, Mandel J, Ewing RE, Efendiev Y, Qin G, Iskandarani M, Coen J, Vodacek A, Kritz M, Haase G (2006) DDDAS approaches to wildland fire modeling and contaminant tracking. In: Proceedings of the 2006 winter simulation conference, Monterey, CA, pp 2117–2124

  17. Douglas CC, Bansal D, Beezley JD, Bennethum LS, Chakraborty S, Coen JL, Efendiev Y, Ewing RE, Hatcher J, Iskandarani M, Johnson CR, Li D, Kim MJ, Lodder RA, Mandel J, Qin G, Vodacek A (2007) Dynamic Data-Driven Application Systems for empty houses, contaminat tracking, and wildland fireline prediction. Grid-Based Probl Solving Environ 239:255–272

    Article  Google Scholar 

  18. Rodriguez R, Cortes A, Margalef T (2009) Injecting dynamic real-time data into a DDDAS for forest fire behavior prediction. In: Computational science—ICCS 2009 (lecture notes in computer science), vol 5545, pp 489–499

  19. Rodriguez R, Cortes A, Margalef T (2010) Towards policies for data insertion in dynamic data driven application systems: a case study sudden changes in wildland fire. In: ICCS 2010—international conference on computational science, proceedings, vol 1, pp 1261–1270

  20. Denham M, Cortes A, Margalef T, Luque E (2008) Applying a dynamic data driven genetic algorithm to improve forest fire spread prediction. In: Computational science—ICCS 2008 (lecture notes in computer science), vol 5103, pp 36–45

  21. Denham M, Cortes A, Margalef T (2009) Computational steering strategy to calibrate input variables in a dynamic data driven genetic algorithm for forest fire spread prediction. In: Computational science—ICCS 2009 (lecture notes in computer science), vol 5545, pp 479–488

  22. Denham M, Cortes A, Margalef T (2009) Parallel dynamic data driven genetic algorithm for forest fire prediction. In: Recent advances in parallel virtual machine and message passing interface, proceedings, vol 5759, pp 323–324

  23. Fujimoto R, Guensler R, Hunter M, Kim H, Lee J, Leonard J, Palekar M, Schwan K, Seshasayee B (2006) Dynamic data driven application simulation of surface transportation systems. Computational science—ICCS 2006 (lecture notes in computer science), vol 3993, pp 425–432

  24. Rossetti M, Hill R, Johansson B, Dunkin A, Ingalls R (2009) A dynamic data-driven approach for rail transport system simulation. In: Proceedings of the 2009 winter simulation conference, Austin, TX, pp 2553–2562

  25. Hunter MP, Fujimoto RM, Suh W, Kim HK (2006) An investigation of real-time dynamic data driven transportation simulation. In: Proceedings of the 2006 winter simulation conference, Monterey, CA, pp 1414–1421

  26. Pingali K, Stodghill P (2004) O’SOAP—a Web services framework for DDDAS applications. In: Computational science—ICCS 2004 (lecture notes in computer science), vol 3038, pp 797–804

  27. Lin S-Y, Chao K-M, Lo C-C (2012) Service-oriented Dynamic Data Driven Application Systems to urban traffic management in resource-bounded environment. ACM SIGAPP Appl Comput Rev 12:35–49

    Article  Google Scholar 

  28. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Google Scholar 

  29. Manju MS, Punithavalli MM (2011) An analysis of Q-Learning algorithms with strategies of reward function. Int J Comput Sci Eng 3:814–820

  30. Vien NA, Viet NH, Lee S, Chung T (2007) Heuristic search based exploration in reinforcement learning. In: Proceedings of the 9th international work conference on artificial neural networks, San Sebasti, Spain, pp 110–118

Download references

Acknowledgments

The author wants to thank the anonymous reviewers for their helpful comments and suggestions. This research work was supported by the Ministry of Science and Technology, Taiwan, Republic of China under the Grant 103-2410-H-033-023.

Author information

Authors and Affiliations

  1. Chung Yuan Christian University, Jhongli, Taiwan

    Szu-Yin Lin

Authors
  1. Szu-Yin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Szu-Yin Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, SY. Reinforcement learning-based prediction approach for distributed Dynamic Data-Driven Application Systems. Inf Technol Manag 16, 313–326 (2015). https://doi.org/10.1007/s10799-014-0205-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10799-014-0205-1

Keywords

Search

Navigation