Skip to main content

Advertisement

SpringerLink
Log in

Data and Decision Intelligence for Human-in-the-Loop Cyber-Physical Systems: Reference Model, Recent Progresses and Challenges

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

With the rapid development of sensing technology, Cyber-Physical Systems (CPS) are connecting our real-world and cyber spaces by real-time situation awareness and intelligent control. In this process, one of the major challenge is how to make fast, accurate and intelligent decisions based on high-dimension, speed and volume sensing data stream. In this paper, we put human into the traditional CPS data process model and formulate a closed-loop computing paradigm for CPS data and decision intelligence. We propose a human-in-the-loop reference model for CPS, which extends the traditional cyber-physical interaction into a closed-loop process based on cyber, physical and human factors. We define the key features of human-in-the-loop CPS, summarize it as three aspects: semantic, interactive, iterative and analyze the major challenges from the perspective of data characteristics. Recent progresses in three typical application domains are reviewed and examined for their decision models and whether they have solved the target issues of human-in-the-loop CPS. According to the review and comparison, the paper finally summarizes several key future opportunities to establish an intelligent human-in-the-loop CPS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

References

  1. Atzori, L., Iera, A., & Morabito, G. (2010). The internet of things: A survey. Computer Networks, 54, 2787–2805.

    Article  MATH  Google Scholar 

  2. Rajkumar, R.R., Lee, I., Sha, L., and Stankovic, J.. (2010) Cyber-physical systems: the next computing revolution. In 47th ACM Design Automation Conference. 731–736.

  3. Sowe, S.K., Simmon, E., Zettsu, K., de Vaulx, F., Bojanova, I.: Cyber-Physical-Human Systems: Putting People in the Loop. IT Professional 18, 10-13 (2016).

  4. Rowley, J. (2007). The wisdom hierarchy: representations of the DIKW hierarchy. Journal of Information Science, 33, 163–180.

    Article  Google Scholar 

  5. Tsai, C.-W., Lai, C.-F., Chiang, M.-C., & Yang, L. T. (2014). Data mining for internet of things: A survey. Communications Surveys & Tutorials, IEEE, 16, 77–97.

    Article  Google Scholar 

  6. Luckham, D. C. (2001). The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Boston: Addison-Wesley Longman Publishing Co., Inc..

    Google Scholar 

  7. Ma, M., Wang, P., Chu, C.-H.. (2013) Data Management for Internet of Things: Challenges, Approaches and Opportunities. In: IEEE International Conference on Internet of Things (iThings). 1144–1151.

  8. Niles, I., Pease, A.. (2001) Towards a standard upper ontology. In: Proceedings of the international conference on Formal Ontology in Information Systems-Volume 2001. 2–9. ACM.

  9. Doerr, M. (2003). The CIDOC conceptual reference module: an ontological approach to semantic interoperability of metadata. AI Magazine, 24, 75.

    Google Scholar 

  10. Li, Z., Chu, C.-H., Yao, W., Behr, R.A.. (2010) Ontology-driven event detection and indexing in smart spaces. In: IEEE International Conference on Semantic Computing (ICSC). 285–292.

  11. Hasan, S., & Curry, E. (2014). Approximate Semantic Matching of Events for the Internet of Things. ACM Transactions on Internet Technology, 14, 1–23.

    Article  Google Scholar 

  12. Cugola, G., & Margara, A. (2012). Processing flows of information: From data stream to complex event processing. ACM Computing Surveys (CSUR), 44, 15.

    Article  Google Scholar 

  13. Dindar, N., Fischer, P.M., Soner, M., Tatbul, N.. (2011) Efficiently correlating complex events over live and archived data streams. In: 5th ACM international conference on Distributed event-based system. 243–254.

  14. Peng, S., Li, Z., Li, Q., Chen, Q., Pan, W., Liu, H., Nie, Y.. (2011) Event detection over live and archived streams. Web-Age Information Management. 566–577. Springer.

  15. Stankovic, J. A. (2014). Research directions for the internet of things. IEEE Internet of Things Journal, 1, 3–9.

    Article  Google Scholar 

  16. Singh, M.P., Hoque, M.A., Tarkoma, S.. (2016) A survey of systems for massive stream analytics. arXiv preprint arXiv:1605.09021.

  17. Krempl, G., Žliobaite, I., Brzeziński, D., Hüllermeier, E., Last, M., Lemaire, V., Noack, T., Shaker, A., Sievi, S., & Spiliopoulou, M. (2014). Open challenges for data stream mining research. ACM SIGKDD explorations newsletter, 16, 1–10.

    Article  Google Scholar 

  18. Kim, H., Choi, B. S., & Huh, M. Y. (2016). Booster in High Dimensional Data Classification. IEEE Transactions on Knowledge and Data Engineering, 28, 29–40.

    Article  Google Scholar 

  19. Zámečníková, E., Kreslíková, J.. (2015) Comparison of platforms for high frequency data processing. In: IEEE 13th International Scientific Conference on Informatics. 296–301.

  20. Mykland, P. A., & Zhang, L. (2012). The econometrics of high frequency data. Statistical Methods for Stochastic Differential Equations, 124, 109.

    Article  MathSciNet  MATH  Google Scholar 

  21. Perera, C., Zaslavsky, A., Christen, P., & Georgakopoulos, D. (2014). Context aware computing for the internet of things: A survey. IEEE Communications Surveys & Tutorials, 16, 414–454.

    Article  Google Scholar 

  22. Ma, M., Wang, P., Chu, C.-H., & Liu, L. (2015). Efficient Multi-Pattern Event Processing over High-Speed Train Data Streams. IEEE Internet of Things Journal, 2(4), 295–309.

    Article  Google Scholar 

  23. Zhou, Q., Simmhan, Y., Prasanna, V.. (2013) Towards hybrid online on-demand querying of realtime data with stateful complex event processing. In: IEEE International Conference on Big Data. 199–205.

  24. Zhang, Y., Qiu, M., Tsai, C.-W., Hassan, M.M., Alamri, A.. (2015) Health-CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Systems Journal.

  25. Leitão, P., Colombo, A. W., & Karnouskos, S. (2016). Industrial automation based on cyber-physical systems technologies: Prototype implementations and challenges. Computers in Industry, 81, 11–25.

    Article  Google Scholar 

  26. Chae, S., Yang, Y., Byun, J., Han, T.D.. (2016) Personal Smart Space: IoT Based User Recognition and Device Control. In: IEEE Tenth International Conference on Semantic Computing (ICSC). 181–182.

  27. Allen, T.. (2016) The NIST Smart Space Project.

  28. Chen, G., Wang, E., Sun, X., & Tang, Y. (2015). An intelligent analysis and mining system for urban lighting information. International Journal of Smart Home, 9, 253–262.

    Article  Google Scholar 

  29. Sim, J.M., Lee, Y., Kwon, O.. (2015) Acoustic sensor based recognition of human activity in everyday life for smart home services. International Journal of Distributed Sensor Networks.

  30. Chetty, G., White, M., & Akther, F. (2015). Smart Phone Based Data Mining for Human Activity Recognition. Procedia Computer Science, 46, 1181–1187.

    Article  Google Scholar 

  31. Azzi, S., Bouzouane, A., Giroux, S., Dallaire, C., Bouchard, B.. (2014) Human activity recognition in big data smart home context. In: IEEE International Conference on Big Data. 1–8.

  32. Moutacalli, M.T., Bouzouane, A., Bouchard, B.. (2014) New frequent pattern mining algorithm tested for activities models creation. In: IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE). 69–76.

  33. Chen, L., Cheung, W.K.. (2014) Recovering Human Mobility Flow Models and Daily Routine Patterns in a Smart Environment. In: IEEE International Conference on Data Mining Workshop (ICDMW). 541–548.

  34. Chen, Y.-C., Peng, W.-C., Huang, J.-L., & Lee, W.-C. (2015). Significant correlation pattern mining in smart homes. ACM Transactions on Intelligent Systems and Technology (TIST), 6, 35.

    Google Scholar 

  35. Cook, D. J., & Krishnan, N. (2014). Mining the home environment. Journal of Intelligent Information Systems, 43, 503–519.

    Article  Google Scholar 

  36. Kulkarni, G., Gode, P., Reddy, J. P., & Deshmukh, M. (2015). Android Based Smart Home System. International Journal of Current Engineering and Technology, 5, 1022–1025.

    Google Scholar 

  37. Bourobou, S. T. M., & Yoo, Y. (2015). User activity recognition in smart homes using pattern clustering applied to temporal ANN algorithm. Sensors, 15, 11953–11971.

    Article  Google Scholar 

  38. Saranya, P., & Thara, L. (2015). Recongnition of Complex Human Activities using visual and Sequence Pattern Mining. International Journal of Research in Computer Applications and Robotics, 3(2), 22–29.

    Google Scholar 

  39. Ma, M., Wang, P., Chu, C.-H.. (2015) LTCEP: Efficient Long-Term Event Processing for Internet of Things Data Streams. In: IEEE International Conference on Internet of Things (iThings). 548–555.

  40. Sussman, J.S.. (2008) Perspectives on intelligent transportation systems (ITS). Springer Science & Business Media.

  41. El Faouzi, N.-E., Leung, H., Kurian, A.: Data fusion in intelligent transportation systems: Progress and challenges–A survey. Information Fusion 12, 4-10 (2011).

  42. Gong, X., Liu, X.. (2003) A data mining based algorithm for traffic network flow forecasting. In: IEEE Intelligent Transportation Systems conference. 193–198.

  43. Tan, H., Wu, Y., Shen, B., Jin, P. J., & Ran, B. (2016). Short-term traffic prediction based on dynamic tensor completion. IEEE Transactions on Intelligent Transportation Systems, 17, 2123–2133.

    Article  Google Scholar 

  44. Nunes, A. A., Dias, T. G., & e Cunha, J. F. (2016). Passenger Journey Destination Estimation From Automated Fare Collection System Data Using Spatial Validation. IEEE Transactions on Intelligent Transportation Systems, 17, 133–142.

    Article  Google Scholar 

  45. Shi, Q., & Abdel-Aty, M. (2015). Big data applications in real-time traffic operation and safety monitoring and improvement on urban expressways. Transportation Research Part C: Emerging Technologies, 58, 380–394.

    Article  Google Scholar 

  46. Ashokkumar, K., Sam, B., & Arshadprabhu, R. (2015). Cloud based intelligent transport system. Procedia Computer Science, 50, 58–63.

    Article  Google Scholar 

  47. Zhang, T., Xia, Y., Zhu, Q., Liu, Y., Shen, J.. (2014) Mining related information of traffic flows on lanes by k-medoids. In: 11th IEEE International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). 390–396.

  48. Necula, E.. (2014) Dynamic traffic flow prediction based on GPS Data. In: IEEE 26th International Conference on Tools with Artificial Intelligence (ICTAI). 922–929.

  49. Shtern, M., Mian, R., Litoiu, M., Zareian, S., Abdelgawad, H., Tizghadam, A.. (2014) Towards a multi-cluster analytical engine for transportation data. In: International Conference on Cloud and Autonomic Computing (ICCAC). 249–257.

  50. Ibrahim, H., Far, B.H.. (2014) Data-oriented intelligent transportation systems. In: IEEE 15th International Conference on Information Reuse and Integration (IRI). 322–329.

  51. Rashid, S., Akram, U., Qaisar, S., Khan, S.A., Felemban, E.. (2014) Wireless sensor network for distributed event detection based on machine learning. In: IEEE International Conference on Internet of Things (iThings). 540–545.

  52. Lin, G., Xin, L., Feng, H., Ying, L.. (2014) A new outlier detection algorithm and its application in intelligent transportation system. In: IEEE 7th Joint International Information Technology and Artificial Intelligence Conference (ITAIC). 442–445.

  53. Foell, S., Phithakkitnukoon, S., Kortuem, G., Veloso, M., & Bento, C. (2015). Predictability of public transport usage: a study of bus rides in Lisbon, Portugal. IEEE Transactions on Intelligent Transportation Systems, 16, 2955–2960.

    Article  Google Scholar 

  54. Miyaji, M.. (2015) Data mining for safety transportation by means of using Internet survey. In: 31st IEEE International Conference on Data Engineering Workshops (ICDEW). 119–123.

  55. Navale, S. A., & Gurav, Y. B. (2015). Crowdedness Spot Acquisition by Using Mobility Based Clustering. International Journal of Science and Research, 4, 171–174.

    Google Scholar 

  56. Lin, Y.X., Wang, P., Ma, M.. (2017) Intelligent Transportation System (ITS): Concept, Challenge and Opportunity. In: IEEE International Conference on High Performance and Smart Computing.

  57. Qiu, M., Gao, W., Chen, M., Niu, J.-W., & Zhang, L. (2011). Energy efficient security algorithm for power grid wide area monitoring system. IEEE Transactions on Smart Grid, 2(4), 715–723.

    Article  Google Scholar 

  58. Gai, K., Qiu, M., Ming, Z., Zhao, H., and Qiu, L.. 2017 Spoofing-Jamming Attack Strategy Using Optimal Power Distributions in Wireless Smart Grid Networks. In: IEEE Transactions on Smart Grid.

  59. Park, S., Ryu, S., Choi, Y., Kim, H.. (2014) A framework for baseline load estimation in demand response: Data mining approach. In: IEEE International Conference on Smart Grid Communications (SmartGridComm). 638–643.

  60. ASGARI, V., Firozyan, M., RADMEHR, M.. (2015) Simultaneous price and demand forecasting in smart power distribution grid. Journal of Selcuk University Natural and Applied Science. 53–62.

  61. Popeangă, J., & Lungu, I. (2014). Forecasting Final Energy Consumption using the Centered Moving Average Method and Time Series Analysis. Database Systems Journal, 5, 42–50.

    Google Scholar 

  62. Ford, V., Siraj, A., Eberle, W.. (2014) Smart grid energy fraud detection using artificial neural networks. In: IEEE Symposium on Computational Intelligence Applications in Smart Grid (CIASG). 1–6.

  63. Tsai, J.C., Yen, N.Y., Hayashi, T.. (2014) Social network based smart grids analysis. In: IEEE International Symposium on Independent Computing (ISIC). 1–6.

  64. Ploennigs, J., Chen, B., Palmes, P., Lloyd, R.. (2014) e2-Diagnoser: A System for Monitoring, Forecasting and Diagnosing Energy Usage. In: IEEE International Conference on Data Mining Workshop (ICDMW). 1231–1234.

  65. Chen, H., Yang, H., Xu, A., Yuan, C.. (2014) A Decision Support System Using Two-Level Classifier for Smart Grid. In: Ninth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC). 42–45.

  66. Kogo, T., Nakamura, S., Pravinraj, S., Arumugam, B.. (2014) A demand side prediction method for persistent scheduled power-cuts in developing countries. In: IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe). 1–6.

  67. Gupta, P.K., Gibtner, A.K., Duchon, M., Koss, D., Schätz, B.. (2015) Using knowledge discovery for autonomous decision making in smart grid nodes. In: IEEE International Conference on Industrial Technology (ICIT). 3134–3139.

  68. Zhen, Z., Wang, F., Sun, Y., Mi, Z., Liu, C., Wang, B., Lu, J.. (2015) SVM based cloud classification model using total sky images for PV power forecasting. In: IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT). 1–5.

  69. Meng Ma, W.L., Zhang, J., Wang, P., Zhou, Y., Liang, X.. (2017) Discover the Fingerprint of Electrical Appliance: Online Appliance Behavior Learning and Detection in Smart Homes. In: International Conference on Ubiquitous Intelligence and Computing.

  70. Makonin, S., Popowich, F., Bartram, L., Gill, B., Bajic, I.V.. (2013) AMPds: A public dataset for load disaggregation and eco-feedback research. In: IEEE Electrical Power & Energy Conference (EPEC). 1–6.

  71. Batra, N., Parson, O., Berges, M., Singh, A., Rogers, A.. (2014) A comparison of non-intrusive load monitoring methods for commercial and residential buildings. arXiv preprint arXiv:1408.6595.

  72. Monacchi, A., Egarter, D., Elmenreich, W., D'Alessandro, S., Tonello, A.M.. (2014) GREEND: An energy consumption dataset of households in Italy and Austria. In: IEEE International Conference on Smart Grid Communications (SmartGridComm). 511–516.

  73. Kolter, J.Z., Johnson, M.J.. (2011) REDD: A public data set for energy disaggregation research. In: Workshop on Data Mining Applications in Sustainability (SIGKDD), San Diego. 59–62.

  74. Kleiminger, W., Beckel, C., Santini, S.. (2015) Household occupancy monitoring using electricity meters. In: ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp). 975–986.

  75. Batra, N., Gulati, M., Singh, A., Srivastava, M.B.. (2013) It's Different: Insights into home energy consumption in India. In: 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings. 1–8.

  76. Kelly, J., Knottenbelt, W.. (2015) The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Scientific Data. 2.

  77. Barker, S., Mishra, A., Irwin, D., Cecchet, E., Shenoy, P., Albrecht, J.. (2012) Smart*: An open data set and tools for enabling research in sustainable homes. SustKDD.

  78. Hu, F., Qiu, M., Li, J., Grant, T., Taylor, D., McCaleb, S., Butler, L., & Hamner, R. (2011). A review on cloud computing: Design challenges in architecture and security. Journal of Computing and Information Technology, 19(1), 25–55.

    Article  Google Scholar 

  79. Li, Y., Gai, K., Ming, Z., Zhao, H., and Qiu, M.. (2016) Intercrossed access controls for secure financial services on multimedia big data in cloud systems. In: ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 12, (4s). 67.

  80. Gai, K., Qiu, M., Zhao, H., Tao, L., & Zong, Z. (2016). Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing. Journal of Network and Computer Applications, 59, 46–54.

    Article  Google Scholar 

  81. Zhu, X., Qin, X., & Qiu, M. (2011). QoS-aware fault-tolerant scheduling for real-time tasks on heterogeneous clusters. IEEE transactions on Computers, 60(6), 800–812.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by National Key R&D Program of China (Grant no.2017YFB1200700), National Natural Science Foundation of China (Grant no.61701007), China Postdoctoral Science Foundation (Grant no.2016M600865) and IBM Shared University Research Project.

Author information

Authors and Affiliations

  1. School of Electronics Engineering and Computer Science, Peking University Peking University, Beijing, 100871, China

    Meng Ma

  2. School of Software and Microelectronics, Peking University, Beijing, 102600, China

    Weilan Lin, Yangxin Lin & Ping Wang

  3. School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School, Shenzhen, 518055, China

    Disheng Pan

  4. National Engineering Research Center for Software Engineering, Peking University, Beijing, 100871, China

    Ping Wang

  5. IBM Research China, Beijing, 100094, China

    Yuchen Zhou & Xiaoxing Liang

Authors
  1. Meng Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weilan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Disheng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yangxin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuchen Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaoxing Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ping Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, M., Lin, W., Pan, D. et al. Data and Decision Intelligence for Human-in-the-Loop Cyber-Physical Systems: Reference Model, Recent Progresses and Challenges. J Sign Process Syst 90, 1167–1178 (2018). https://doi.org/10.1007/s11265-017-1304-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-017-1304-0

Keywords

Search

Navigation