skip to main content
10.1145/3457913.3457919acmotherconferencesArticle/Chapter ViewAbstractPublication PagesinternetwareConference Proceedingsconference-collections
research-article

Simulated or Physical? An Empirical Study on Input Validation for Context-aware Systems in Different Environments

Published: 21 July 2021 Publication History

Abstract

Context-Aware Systems (a.k.a. CASs) integrate cyber and physical space to provide context-aware adaptive functionalities. Building context-aware systems is challenging due to the uncertainty of the real physical environment. Therefore, input validation for context-aware systems plays a significant role in keeping the systems executing safely. Input validation approaches have been proposed to monitor and guard the executions of context-aware systems. However, few of these works (17%, 2 out of 12) evaluated their approaches with a real context-aware system in a real physical environment. In this paper, we study and compare the effectiveness of input validation approaches for context-aware system in both a simulated and a physical environment. We built a testing platform, RM-Testing, based on DJI RoboMaster S1 robot car. We implemented three up-to-date input validation approaches, and evaluated their effectiveness in improving the success rate of the robot car’s executions. The results show that the selected input validation approaches are effective in guarantee the safe execution of context-aware systems, which improve the success rate by 82% in the simulated environment, and 50% in the physical environment. However, the effectiveness of these approaches does vary in different environment. Thus, we believe that such CASs-based input validation works should be evaluated in the physical environment to better validate their effectiveness and usefulness.

References

[1]
[1] Lü J, Ma X, Huang Y, et al. Internetware: a shift of software paradigm[C]//Proceedings of the First Asia-Pacific Symposium on Internetware. 2009: 1-9.
[2]
[2] https://www.dji.com/cn/robomaster-s1.
[3]
[3] https://unity.com.
[4]
[4] https://link.springer.com/chapter/10.1007/978-3-030-00262-6_11.
[5]
[5] https://www.bbc.com/news/technology-48308852.
[6]
[6] National Transportation Safety Committee of Indonesian, “Preliminary Report on the Lion Air B737 MAX 8 accident,”, https://reports.aviation-safety.net/2018/20181029-0_B38M_PK-LQP_PRELIMINARY.pdf.
[7]
[7] Xi W, Xu C, Yang W, et al. How context inconsistency and its resolution impact context-aware applications[J]. Journal of Frontiers of Computer Science and Technology, 2014, 8(4): 427.
[8]
[8] Esfahani N, Malek S. Uncertainty in self-adaptive software systems[M]//Software Engineering for Self-Adaptive Systems II. Springer, Berlin, Heidelberg, 2013: 214-238.
[9]
[9] Huiyan Wang, Jingwei Xu, Chang Xu, Xiaoxing Ma, and Jian Lu. DISSECTOR: Input Validation for Deep Learning Applications by Crossing-layer Dissection. In Proceedings of the 42nd ACM/IEEE International Conference on Software Engineering (ICSE 2020), pp. 727-738, Seoul, South Korea, May 2020.
[10]
[10] Nentwich C, Emmerich W, Finkelsteiin A, et al. Flexible consistency checking[J]. ACM Transactions on Software Engineering and Methodology (TOSEM), 2003, 12(1): 28-63.
[11]
[11] Xu C, Cheung S C, Chan W K, et al. Partial constraint checking for context consistency in pervasive computing[J]. ACM Transactions on Software Engineering and Methodology (TOSEM), 2010, 19(3): 1-61.
[12]
[12] Egyed A. Fixing inconsistencies in UML design models[C]//29th International Conference on Software Engineering (ICSE’07). IEEE, 2007: 292-301.
[13]
[13] Gehrke J, Madden S. Query processing in sensor networks[J]. IEEE Pervasive computing, 2004, 3(1): 46-55.
[14]
[14] Tarr P, Clarke L A. Consistency management for complex applications[C]//Proceedings of the 20th international conference on Software engineering. IEEE, 1998: 230-239.
[15]
[15] Xu C, Cheung S C, Ma X, et al. Adam: Identifying defects in context-aware adaptation[J]. Journal of Systems and Software, 2012, 85(12): 2812-2828.
[16]
[16] Ernst M D, Perkins J H, Guo P J, et al. The Daikon system for dynamic detection of likely invariants[J]. Science of computer programming, 2007, 69(1-3): 35-45.
[17]
[17] Jiang H. Invariant Inferring and Monitoring in Robotic Systems[J]. 2014.
[18]
[18] Aliabadi, M. R., Kamath, A. A., Gascon-Samson, J. and Pattabiraman, K., “ARTINALI: dynamic invariant detection for cyber-physical system security,” in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, 2017, pp. 349-361.
[19]
[19] Y. Qin, T. Xie, C. Xu, A. Astoga and J. Lu, “CoMID: context-based multi-invariant detection for monitoring cyber-physical software,” in IEEE Transactions on Reliability, to be published, 2019.
[20]
[20] Nentwich C, Capra L, Emmerich W, et al. xlinkit: A consistency checking and smart link generation service[J]. ACM Transactions on Internet Technology (TOIT), 2002, 2(2): 151-185.
[21]
[21] Ramirez A J, Jensen A C, Cheng B H C, et al. Automatically exploring how uncertainty impacts behavior of dynamically adaptive systems[C]//2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). IEEE, 2011: 568-571.
[22]
[22] Pei K, Cao Y, Yang J, et al. Deepxplore: Automated whitebox testing of deep learning systems[C]//proceedings of the 26th Symposium on Operating Systems Principles. 2017: 1-18.
[23]
[23] Chu X, Morcos J, Ilyas I F, et al. Katara: A data cleaning system powered by knowledge bases and crowdsourcing[C]//Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. 2015: 1247-1261.
[24]
[24] Kulkarni D, Tripathi A. A framework for programming robust context-aware applications[J]. IEEE Transactions on Software Engineering, 2010, 36(2): 184-197.
[25]
[25] Sama M, Elbaum S, Raimondi F, et al. Context-aware adaptive applications: Fault patterns and their automated identification[J]. IEEE Transactions on Software Engineering, 2010, 36(5): 644-661.
[26]
[26] Wang Z, Elbaum S, Rosenblum D S. Automated generation of context-aware tests[C]//29th International Conference on Software Engineering (ICSE’07). IEEE, 2007: 406-415.
[27]
[27] Tian Y, Pei K, Jana S, et al. Deeptest: Automated testing of deep-neural-network-driven autonomous cars[C]//Proceedings of the 40th international conference on software engineering. 2018: 303-314.

Cited By

View all
  • (2022)Simulation Might Change Your Results: A Comparison of Context-Aware System Input Validation in Simulated and Physical EnvironmentsJournal of Computer Science and Technology10.1007/s11390-021-1669-137:1(83-105)Online publication date: 1-Feb-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
Internetware '20: Proceedings of the 12th Asia-Pacific Symposium on Internetware
November 2020
264 pages
ISBN:9781450388191
DOI:10.1145/3457913
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. context-aware systems
  2. input validation
  3. self-driving cars
  4. testing infrastructure

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

Internetware'20
Internetware'20: 12th Asia-Pacific Symposium on Internetware
November 1 - 3, 2020
Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 55 of 111 submissions, 50%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Simulation Might Change Your Results: A Comparison of Context-Aware System Input Validation in Simulated and Physical EnvironmentsJournal of Computer Science and Technology10.1007/s11390-021-1669-137:1(83-105)Online publication date: 1-Feb-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media