Skip to main content
SpringerLink
Log in

Survey of Testing Methods and Testbed Development Concerning Internet of Things

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

The concept of Internet of Things (IoT) was designed to change everyday lives of people via multiple forms of computing and easy deployment of applications. In recent years, the increasing complexity of IoT-ready devices and processes has led to potential risks related to system reliability. Therefore, the comprehensive testing of IoT technology has attracted the attention of many researchers, which promotes the extensive development of IoT testing methods and infrastructure. However, the current research on IoT testing methods and testbeds mainly focuses on specific application scenarios, lacking systematic review and analysis of many applications from different points of view. This paper systematically summarizes the latest testing methods covering different IoT fields and discusses the development status of the existing Internet of things testbed. Findings of this review demonstrate that IoT testing is moving toward larger scale and intelligent testing, and that in near future, IoT test architecture is set to become more standardized and universally applicable with multi-technology convergence—i.e., a combination of big data, cloud computing, and artificial intelligence—being the prime focus of IoT testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Madakam, S., Lake, V., Lake, V., & Lake, V. (2015). Internet of Things (IoT): A literature review. Journal of Computer and Communications, 3(05), 164.

    Article  Google Scholar 

  2. Shah, S. H., & Yaqoob, I. (2016) A survey: Internet of Things (IOT) technologies, applications and challenges. In 2016 ieee smart energy grid engineering (SEGE), 2016, pp. 381–385: IEEE.

  3. Ghourab, E. M., Azab, M., Rizk, M., & Mokhtar, A. (2017). Security versus reliability study for power-limited mobile IoT devices. In 2017 8th IEEE annual information technology, electronics and mobile communication conference (IEMCON), 2017, pp. 430–438: IEEE.

  4. Maalel, N., Natalizio, E., Bouabdallah, A., Roux, P., & Kellil, M. (2013) Reliability for emergency applications in internet of things. In 2013 IEEE international conference on distributed computing in sensor systems, 2013, pp. 361–366: IEEE.

  5. Hossain, M. S., Muhammad, G., Abdul, W., Song, B., & Gupta, B. B. (2018). Cloud-assisted secure video transmission and sharing framework for smart cities. Future Generation Computer System, 83, 596–606.

    Article  Google Scholar 

  6. Wu, L., Zhang, Y., Choo, K.-K.R., & He, D. (2017). Efficient identity-based encryption scheme with equality test in smart city. IEEE Transaction Sustainable Computer, 3(1), 44–55.

    Article  Google Scholar 

  7. Tao, M., Zuo, J., Liu, Z., Castiglione, A., & Palmieri, F. (2018). Multi-layer cloud architectural model and ontology-based security service framework for IoT-based smart homes. Future Generation Computer System, 78, 1040–1051.

    Article  Google Scholar 

  8. Suryadevara, N. K. (2017). Wireless sensor sequence data model for smart home and IoT data analytics. In Proceedings of the first international conference on computational intelligence and informatics, 2017, pp. 441–447: Springer.

  9. Mijić, D., & Varga, E. (2018). Unified iot platform architecture platforms as major iot building blocks. In 2018 International conference on computing and network communications (CoCoNet), 2018, pp. 6–13: IEEE.

  10. Manashty, A., & Light, J. (2019). Life model: A novel representation of life-long temporal sequences in health predictive analytics. Future Genereration Computer System, 92, 141–156.

    Article  Google Scholar 

  11. Elhoseny, M., Ramírez-González, G., Abu-Elnasr, O. M., Shawkat, S. A., Arunkumar, N., & Farouk, A. (2018). Secure medical data transmission model for IoT-based healthcare systems. IEEE Access, 6, 20596–20608.

    Article  Google Scholar 

  12. Zhang, Y., Wang, W., Wu, N., & Qian, C. (2015). IoT-enabled real-time production performance analysis and exception diagnosis model. IEEE Transactions on Automation Science and Engineering, 13(3), 1318–1332.

    Article  Google Scholar 

  13. Mars, D., Gammar, S. M., Lahmadi, A., & Saidane, L. A. (2019). Using information centric networking in internet of things: A survey. Wireless Personal Communications, 105(1), 87–103.

    Article  Google Scholar 

  14. El-Hajj, M., Fadlallah, A., Chamoun, M., & Serhrouchni, A. (2019). A survey of internet of things (IoT) authentication schemes. Sensors, 19(5), 1141.

    Article  Google Scholar 

  15. Mouradian, C., Naboulsi, D., Yangui, S., Glitho, R. H., Morrow, M. J., & Polakos, P. A. (2017). A comprehensive survey on fog computing: State-of-the-art and research challenges. IEEE Communications Surveys & Tutorials, 20(1), 416–464.

    Article  Google Scholar 

  16. Abbas, N., Zhang, Y., Taherkordi, A., & Skeie, T. (2017). Mobile edge computing: A survey. IEEE Internet of Things Journal, 5(1), 450–465.

    Article  Google Scholar 

  17. Murad, G., Badarneh, A., Qusef, A., & Almasalha, F. (2018). Software testing techniques in iot. In 2018 8th International conference on computer science and information technology (CSIT), 2018, pp. 17–21: IEEE.

  18. Wu, J., Jiang, W., Mei, Y., Zhou, Y., & Wang, T. (2018). A survey on the progress of testing techniques and methods for wireless sensor networks. IEEE Access, 7, 4302–4316.

    Article  Google Scholar 

  19. Xie, W., Jiang, Y., Tang, Y., Ding, N., & Gao, Y. (2017). Vulnerability detection in iot firmware: A survey. In 2017 IEEE 23rd International conference on parallel and distributed systems (ICPADS), 2017, pp. 769–772: IEEE.

  20. Bures, M., Cerny, T., & Ahmed, B. S. (2018). Internet of things: Current challenges in the quality assurance and testing methods. In International conference on information science and applications, 2018, pp. 625–634: Springer.

  21. Rosenkranz, P., Wählisch, M., Baccelli, E., & Ortmann, L. (2015). A distributed test system architecture for open-source IoT software. In Proceedings of the 2015 Workshop on IoT challenges in Mobile and Industrial Systems, 2015, pp. 43–48.

  22. Bures, M., Klima, M., Rechtberger, V., Bellekens, X., Tachtatzis, C., Atkinson, R., et al. (2020). Interoperability and integration testing methods for IoT systems: A systematic mapping study. In International conference on software engineering and formal methods, 2020, pp. 93–112: Springer.

  23. Dias, J. P., Couto, F., Paiva, A. C., & Ferreira, H. S. (2018). A brief overview of existing tools for testing the internet-of-things. In 2018 IEEE international conference on software testing, verification and validation workshops (ICSTW), 2018, pp. 104–109: IEEE.

  24. Lee, I., & Lee, K. (2015). The Internet of Things (IoT): Applications, investments, and challenges for enterprises. Business Horizons, 58(4), 431–440.

    Article  Google Scholar 

  25. Pflanzner, T., & Kertész, A. (2016). A survey of IoT cloud providers. In 2016 39th International convention on information and communication technology, electronics and microelectronics (MIPRO), 2016, pp. 730–735: IEEE.

  26. Stergiou, C., Psannis, K. E., Kim, B.-G., & Gupta, B. (2018). Secure integration of IoT and cloud computing. Future Generation Computer System, 78, 964–975.

    Article  Google Scholar 

  27. Dizdarević, J., Carpio, F., Jukan, A., & Masip-Bruin, X. (2019). A survey of communication protocols for internet of things and related challenges of fog and cloud computing integration. ACM Computing Surveys (CSUR), 51(6), 1–29.

    Article  Google Scholar 

  28. Al-Garadi, M. A., Mohamed, A., Al-Ali, A. K., Du, X., Ali, I., & Guizani, M. (2020). A survey of machine and deep learning methods for internet of things (IoT) security. IEEE Communications Surveys & Tutorials, 22(3), 1646–1685.

    Article  Google Scholar 

  29. Verma, S., Kawamoto, Y., Fadlullah, Z. M., Nishiyama, H., & Kato, N. (2017). A survey on network methodologies for real-time analytics of massive IoT data and open research issues. IEEE Communications Surveys & Tutorials, 19(3), 1457–1477.

    Article  Google Scholar 

  30. Zhang, Y., Ren, J., Liu, J., Xu, C., Guo, H., & Liu, Y. (2017). A survey on emerging computing paradigms for big data. Chinese Journal of Electronics, 26(1), 1–12.

    Article  Google Scholar 

  31. Khan, I., Belqasmi, F., Glitho, R., Crespi, N., Morrow, M., & Polakos, P. (2015). Wireless sensor network virtualization: A survey. IEEE Communications Surveys & Tutorials, 18(1), 553–576.

    Article  Google Scholar 

  32. Guillén, E., Sánchez, J., & López, L. R. (2017). IoT protocol model on healthcare monitoring. In VII Latin American congress on biomedical engineering CLAIB 2016, bucaramanga, santander, Colombia, October 26th-28th, 2016, 2017, pp. 193–196: Springer.

  33. Cui, K., Zhou, K., Qiu, T., Li, M., & Yan, L. (2017). A hierarchical combinatorial testing method for smart phone software in wearable IoT systems. Computers & Electrical Engineering, 61, 250–265.

    Article  Google Scholar 

  34. Cao, Q. H., Khan, I., Farahbakhsh, R., Madhusudan, G., Lee, G. M., & Crespi, N. (2016). A trust model for data sharing in smart cities. In 2016 IEEE International conference on communications (ICC), 2016, pp. 1–7: IEEE.

  35. Mahfuz, S., Isah, H., Zulkernine, F., & Nicholls, P. ()2018. Detecting irregular patterns in IoT streaming data for fall detection. In 2018 IEEE 9th annual information technology, electronics and mobile communication conference (IEMCON), 2018, pp. 588–594: IEEE.

  36. Maldonado, F. J., Selmic, R. R., & Figueroa, F. (2018). Health Electronic Data Sheet (HEDS) for enhanced transducer monitoring, reliability and safe operation. In 2018 IEEE international instrumentation and measurement technology conference (I2MTC), 2018, pp. 1–6: IEEE.

  37. Satija, U., Ramkumar, B., & Manikandan, M. S. (2017). Real-time signal quality-aware ECG telemetry system for IoT-based health care monitoring. IEEE Internet of Things Journal, 4(3), 815–823.

    Article  Google Scholar 

  38. Lee, J., Debnath, M., Patki, A., Hasan, M., & Nicopoulos, C. (2018). Hardware-based online self-diagnosis for faulty device identification in large-scale IoT systems. In 2018 IEEE/ACM third international conference on internet-of-things design and implementation (IoTDI), 2018, pp. 96–104: IEEE.

  39. Shahid, M. R., Blanc, G., Zhang, Z., & Debar, H. (2018). IoT devices recognition through network traffic analysis. In 2018 IEEE international conference on big data (big data), 2018, pp. 5187–5192: IEEE.

  40. Meidan, Y., Bohadana, M., Shabtai, A., Ochoa, M., Ole Tippenhauer,N., Davis Guarnizo, J., et al. (2017). Detection of unauthorized IoT devices using machine learning techniques." http://arxiv.org/abs/170904647 arXiv preprint.

  41. Bezawada, B., Bachani, M., Peterson, J., Shirazi, H., Ray, I., & Ray, I. (2018). Behavioral fingerprinting of iot devices. In Proceedings of the 2018 workshop on attacks and solutions in hardware security, 2018, pp. 41–50.

  42. Nguyen, T. D., Marchal, S., Miettinen, M., Dang, M. H., Asokan, N., & Sadeghi, A.-R. (2018). Dïot: A crowdsourced self-learning approach for detecting compromised iot devices. CoRR.

  43. Lee, H. (2017). Framework and development of fault detection classification using IoT device and cloud environment. Journal of Manufacturing Systems, 43, 257–270.

    Article  Google Scholar 

  44. Lv, F., Wen, C., Bao, Z., & Liu, M. (2016). Fault diagnosis based on deep learning. In 2016 American control conference (ACC), 2016, pp. 6851–6856: IEEE.

  45. Lee, S.-Y., Wi, S.-r., Seo, E., Jung, J.-K., & Chung, T.-M. (2017). ProFiOt: Abnormal Behavior Profiling (ABP) of IoT devices based on a machine learning approach. In 2017 27th International telecommunication networks and applications conference (ITNAC), 2017, pp. 1–6: IEEE.

  46. Cui, L., Yang, S., Chen, F., Ming, Z., Lu, N., & Qin, J. (2018). A survey on application of machine learning for Internet of Things. International Journal of Machine Learning and Cybernetics, 9(8), 1399–1417.

    Article  Google Scholar 

  47. Xiao, L., Wan, X., Lu, X., Zhang, Y., & Wu, D. (2018). IoT security techniques based on machine learning: How do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 35(5), 41–49.

    Article  Google Scholar 

  48. Syafrudin, M., Alfian, G., Fitriyani, N. L., & Rhee, J. (2018). Performance analysis of IoT-based sensor, big data processing, and machine learning model for real-time monitoring system in automotive manufacturing. Sensors, 18(9), 2946.

    Article  Google Scholar 

  49. Babar, M., & Arif, F. (2017). Smart urban planning using big data analytics to contend with the interoperability in Internet of Things. Future Gener. Comput. Syst., 77, 65–76.

    Article  Google Scholar 

  50. Din, S., & Paul, A. (2019). Retracted: Smart health monitoring and management system: Toward autonomous wearable sensing for internet of things using big data analytics. Elsevier.

    Book  Google Scholar 

  51. Queiroz, J., Barbosa, J., Dias, J., Leitão, P., & Oliveira, E. (2017). Development of a smart electric motor testbed for Internet of Things and big data technologies. In IECON 2017–43rd Annual conference of the ieee industrial electronics society, 2017, pp. 3435–3440: IEEE.

  52. Rathore, M. M., Ahmad, A., Paul, A., & Rho, S. (2016). Urban planning and building smart cities based on the internet of things using big data analytics. Comput. Networks, 101, 63–80.

    Article  Google Scholar 

  53. Ahsan, U., & Bais, A. (2016). A review on big data analysis and Internet of Things. In 2016 IEEE 13th International conference on mobile Ad Hoc and sensor systems (MASS), 2016, pp. 325–330: IEEE.

  54. Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Abaker Targio Hashem, I., Siddiqa, A., et al. (2017). Big IoT data analytics: Architecture, opportunities, and open research challenges. IEEE Access, 5, 5247–5261.

    Article  Google Scholar 

  55. Siregar, B., Nasution, A. B. A., & Fahmi, F. (2016). Integrated pollution monitoring system for smart city. In 2016 International conference on ICT For smart society (ICISS), 2016, pp. 49–52: IEEE.

  56. Meng, X. (2018). Research on the inspection and monitoring system of power equipment based on cloud storage platform. In 2018 Chinese control and decision conference (CCDC), 2018, pp. 1567–1571: IEEE.

  57. Hu, F., & Shao, L. (2017). Design of remote irrigation system in farmland based on the cloud platform. In 2017 29th Chinese control And Decision Conference (CCDC), 2017, pp. 1125–1129: IEEE.

  58. Radogna, A. V., Capone, S., Anna Di Lauro, G., Fiore, N., Longo, V., Giampetruzzi, L., et al. (2018). A smart breath analyzer for monitoring home mechanical ventilated patients. In Convegno Nazionale Sensori, 2018, pp. 465–471: Springer.

  59. Madhan, E. (2018). Pharmacovigilance predictive analysis using NLP-based cloud. International Journal of Biomedical Engineering and Technology, 26(3–4), 316–324.

    Article  Google Scholar 

  60. Mora, N., Matrella, G., & Ciampolini, P. (2018). Cloud-based behavioral monitoring in smart homes. Sensors, 18(6), 1951.

    Article  Google Scholar 

  61. Chifor, B.-C., Bica, I., Patriciu, V.-V., & Pop, F. (2018). A security authorization scheme for smart home Internet of Things devices. Future Generation Computer System, 86, 740–749.

    Article  Google Scholar 

  62. Huda, S., Miah, S., Yearwood, J., Alyahya, S., Al-Dossari, H., & Doss, R. (2018). A malicious threat detection model for cloud assisted internet of things (CoT) based industrial control system (ICS) networks using deep belief network. Journal of Parallel Distributed Computing, 120, 23–31.

    Article  Google Scholar 

  63. Zuo, Y., Tao, F., & Nee, A. Y. (2018). An Internet of things and cloud-based approach for energy consumption evaluation and analysis for a product. International Journal of Computer Integrated Manufacturing, 31(4–5), 337–348.

    Article  Google Scholar 

  64. Rafferty, L., Iqbal, F., Aleem, S., Lu, Z., Huang, S.-C., & Hung, P. C. (2018). Intelligent multi-agent collaboration model for smart home IoT security. In 2018 IEEE international congress on internet of things (ICIOT), 2018, pp. 65–71: IEEE.

  65. Bishop, H. L., Wang, P., Fan, D., Lach, J., & Calhoun, B. H. (2018). Lighting IoT test environment (LITE) platform: Evaluating light-powered, energy harvesting embedded systems. In 2018 Global Internet of Things Summit (GIoTS), 2018, pp. 1–6: IEEE.

  66. Ham, Y.-h., Jung, H.-t., Kim, H.-c., & Chung, J.-w. (2017). A Study on OPNET state machine model based IoT network layer test. In International conference on information science and applications, 2017, pp. 38–45: Springer.

  67. Nowak, S., Tehrani, N., Metcalfe, M. S., Eberle, W., & Wang, L. (2018). Cloud-based DERMS test platform using real-time power system simulation. In 2018 IEEE power & energy society general meeting (PESGM), 2018, pp. 1–5: IEEE.

  68. Terroso-Saenz, F., González-Vidal, A., Ramallo-González, A. P., & Skarmeta, A. F. (2019). An open IoT platform for the management and analysis of energy data. Future Generation Computer System, 92, 1066–1079.

    Article  Google Scholar 

  69. Latre, S., Leroux, P., Coenen, T., Braem, B., Ballon, P., & Demeester, P. (2016). City of things: An integrated and multi-technology testbed for IoT smart city experiments. In 2016 IEEE international smart cities conference (ISC2), 2016, pp. 1–8: IEEE.

  70. Siboni, S., et al. (2019). Security testbed for Internet-of-Things devices. IEEE Transactions on Reliability, 68(1), 23–44.

    Article  Google Scholar 

  71. Matheu-García, S. N., Hernández-Ramos, J. L., Skarmeta, A. F., & Baldini, G. (2019). Risk-based automated assessment and testing for the cybersecurity certification and labelling of IoT devices. Computer Standards & Interfaces, 62, 64–83.

    Article  Google Scholar 

  72. Siboni, S., Shabtai, A., Tippenhauer, N. O., Lee, J., & Elovici, Y. (2016). Advanced security testbed framework for wearable IoT devices. ACM Transactions on Internet Technology (TOIT), 16(4), 1–25.

    Article  Google Scholar 

  73. Lally G., & Sgandurra, D. (2018) Towards a framework for testing the security of IoT devices consistently. In International workshop on emerging technologies for authorization and authentication, 2018, pp. 88–102: Springer.

  74. Lanza, J., Sanchez, L., Santana, J. R., Agarwal, R., Ni Kefalakis, P., Grace, T Elsaleh, Zhao, M., Tragos, E., Nguyen, H., Cirillo, F., Steinke, R., & Soldatos, J. (2018). Experimentation as a service over semantically interoperable Internet of Things testbeds. IEEE Access, 6, 51607–51625.

    Article  Google Scholar 

  75. Datta, S. K., Bonnet, C., Baqa, H., Zhao, M., & Le-Gall, F. (2018). Approach for semantic interoperability testing in internet of things. In 2018 Global internet of things summit (GIoTS), 2018, pp. 1–6: IEEE.

  76. Zhao, M., Kefalakis, N., Grace, P., Soldatos, J., Le-Gall, & Cousin, P. (2016). Towards an interoperability certification method for semantic federated experimental iot testbeds. In International Conference on Testbeds and Research Infrastructures, 2016, pp. 103–113: Springer.

  77. Ziegler, S., Fdida, S., Viho, C., & Watteyne, T. (2016). F-interop–online platform of interoperability and performance tests for the internet of things. In Interoperability, safety and security in IoT: Springer, 2016, pp. 49–55.

  78. Zorian,Y. (2015). Keynote 3:" Ensuring robustness in today's IoT era. In 2015 10th International design & test symposium (IDT), 2015, pp. 1–1: IEEE.

  79. Jia, B., Hao, L., Zhang, C., & Chen, D. (2018). A dynamic estimation of service level based on fuzzy logic for robustness in the internet of things. Sensors, 18(7), 2190.

    Article  Google Scholar 

  80. Brady, S., Hava, A., Perry, P., Murphy, J., Magoni, D., & Portillo-Dominguez, A. O. (2017). Towards an emulated IoT test environment for anomaly detection using NEMU. In 2017 Global Internet of Things Summit (GIoTS), 2017, pp. 1–6: IEEE.

  81. Lin, W., Zeng, H., Gao, H., Miao, H., & Wang, X. (2018). Test sequence reduction of wireless protocol conformance testing to internet of things. Security and Communication Networks, 2018(1), 2018.

    Google Scholar 

  82. Yushev, A., Schappacher, M., & Sikora, A. (2016). "Titan TTCN-3 based test framework for resource constrained systems. MATEC Web of Conferences, 75, 06005.

    Article  Google Scholar 

  83. Wang, N., Xu, Y., Zhang, Y., Liu, M., Shen, D., & Wang, H. (2018). An algorithm to calculate phase and amplitude of tag on RFID protocol conformance test system. In 2018 20th international conference on advanced communication technology (ICACT), pp. 120–126: IEEE.

  84. Schieferdecker,I., Kretzschmann, S., Rennoch, A., & Wagner, M. (2017). IoT-testware-an eclipse project. In 2017 IEEE international conference on software quality, reliability and security (QRS), pp. 1–8: IEEE.

  85. Kim, H., Ahmad, A., Hwang, J., Baqa, H., Gall, F. L., Ortega, M. A. R., & Song, J. S. (2018). IoT-TaaS: Towards a prospective IoT testing framework. IEEE Access, 6, 15480–15493.

    Article  Google Scholar 

  86. Ahmad, A., Bouquet, F., Fourneret, E., & Legeard, B. (2018). Model-based testing for internet of things systems. Advances in Computers, 108, 1–58.

    Article  Google Scholar 

  87. Cao, P., Badger, E. C., Kalbarczyk, Z. T., Iyer, R. K., Withers, A., & Slagell, A. J. (2015). Towards an unified security testbed and security analytics framework. In Proceedings of the 2015 symposium and bootcamp on the science of security, 2015, pp. 1–2.

  88. Samant, S. S., Chhetri, M. B., Vo, Q. B., Kowalczyk, R., & Nepal, S. (2017). Towards quality-assured data delivery in cloud-based iot platforms for smart cities. In 2017 IEEE 3rd international conference on collaboration and internet computing (CIC), 2017, pp. 291–298: IEEE.

  89. Tushar, W., Yuen, C., Chai, B., Huang, S., Wood, K. L., Kerk, S. G., & Yang, Z. (2016). Smart grid testbed for demand focused energy management in end user environments. IEEE Wireless Commun., 23(6), 70–80.

    Article  Google Scholar 

  90. Kim,Y.-D., Jung, S.-H., Gu, D.-Y., Kim, H.-K., & Song, C.-H. (2017). Iot sensor based mobility performance test-bed for disaster response robots. In 2017 6th IIAI international congress on advanced applied informatics (IIAI-AAI), pp. 990–991: IEEE.

  91. Srbinovski, B., Conte, G., Morrison, A. P., Leahy, P., & Popovici, E., (2017). ECO: An IoT platform for wireless data collection, energy control and optimization of a miniaturized wind turbine cluster: Power analysis and battery life estimation of IoT platform. In 2017 IEEE international conference on industrial technology (ICIT), pp. 412–417: IEEE.

  92. Maurin, T., Ducreux, L.-F., Caraiman, G. & Sissoko, P. (2018). IoT security assessment through the interfaces P-SCAN test bench platform. In 2018 Design, automation & test in europe conference & exhibition (DATE), pp. 1007–1008: IEEE.

  93. Mäkinen, A., Jiménez, J., & Morabito, R. (2017). ELIoT: Design of an emulated IoT platform. In 2017 IEEE 28th annual international symposium on personal, indoor, and mobile radio communications (PIMRC), p. 1–7: IEEE.

  94. Berouine, A., Lachhab, F., Malek, Y. N., Bakhouya, M., & Ouladsine, R. (2017). A smart metering platform using big data and IoT technologies. In 2017 3rd International conference of cloud computing technologies and applications (CloudTech), pp. 1–6: IEEE.

  95. Le-Trung, Q. (2017). Towards an IoT network testbed emulated over OpenStack cloud infrastructure. In 2017 International conference on recent advances in signal processing, telecommunications & computing (SigTelCom), pp. 246–251: IEEE.

  96. Bounceur, A., Marc, O., Lounis, M., Soler, J., Clavier, L., Combeau, P., et al. (2018). Cupcarbon-lab: An iot emulator. In 2018 15th IEEE annual consumer communications & networking conference (CCNC), pp. 1–2: IEEE.

  97. Flauzac, O., Gonzalez, C., & Nolot, F. (2016). Developing a distributed software defined networking testbed for IoT. Procedia Comput. Sci., 83, 680–684.

    Article  Google Scholar 

  98. Dezfouli, B., Amirtharaj, I., & Li, C.-C.C. (2018). EMPIOT: An energy measurement platform for wireless IoT devices. Journal of Network and Computer Applications, 121, 135–148.

    Article  Google Scholar 

  99. Huh, J. H., Kim, D. H., & Kim, J.-D. (2017). Newsbed: The internet of things testbed platform. In 2017 International conference on information networking (ICOIN), pp. 492–494: IEEE.

  100. Costantino D., Malagnini, G., Carrera, F., Rizzardi, A., Boccadoro, P., Sicari, S., et al. (2018). Solving interoperability within the smart building: A real test-bed. In 2018 ieee international conference on communications workshops (ICC Workshops), pp. 1–6: IEEE.

  101. Fleury, E., Mitton, N., Noel, T., & Adjih, C. (2015). Fit iot-lab: The largest iot open experimental testbed. Ercim News, 101, 4.

    Google Scholar 

  102. Belli, L., et al. (2015). Design and deployment of an IoT application-oriented testbed. Computer, 48(9), 32–40.

    Article  Google Scholar 

  103. Marinissen, E. J., Zorian, Y., Konijnenburg, M., Huang, C.-T., Hsieh, P., Cockburn, P., et al., (2016). IoT: Source of test challenges. In 2016 21th IEEE European test symposium (ETS), pp. 1–10: IEEE.

  104. Kecskemeti, G., Casale, G., Jha, D. N., Lyon, J., & Ranjan, R. (2017). Modelling and simulation challenges in internet of things. IEEE Cloud Comput., 4(1), 62–69.

    Article  Google Scholar 

  105. Lunardi, W. T., de Matos, E., Tiburski, R., Amaral, L. A., Marczak, S., & Hessel, F. (2015). Context-based search engine for industrial IoT: Discovery, search, selection, and usage of devices. In 2015 IEEE 20th Conference on emerging technologies & factory automation (ETFA), pp. 1–8: IEEE.

  106. Rausch, M., Lämmer, S., & Treiber, M. (2018). Self-healing road networks: a self-organized management strategy for traffic incidents in urban road networks. arXiv preprint http://arxiv.org/abs/181111300.

  107. Aktas, M. S., & Astekin, M. (2019). Provenance aware run-time verification of things for self-healing Internet of Things applications. Concurrency and Computation: Practice and Experience, 31(3), e4263.

    Article  Google Scholar 

  108. Estebsari, A., Orlando, M., Pons, E., Acquaviva, A., & Patti, E. (2018). A novel Internet-of-Things infrastructure to support self-healing distribution systems. In 2018 International Conference on Smart Energy Systems and Technologies (SEST), pp. 1–6: IEEE.

  109. Ding, M., Harpe, P., Chen, G., Busze, B., Liu, Y.-H., Bachmann, C., Philips, K., & Roermund, A. (2018). A hybrid design automation tool for SAR ADCs in IoT. IEEE Transactions on Very Large Scale Integration Systems, 206(12), 2853–2862.

    Article  Google Scholar 

  110. Hamalainen, M., & Tyrvainen, P. (2016). A framework for IoT service experiment platforms in smart-city environments. In 2016 IEEE International Smart Cities Conference (ISC2), pp. 1–8: IEEE.

  111. Bedhief, I., Kassar, M., & Aguili, T. (2016). SDN-based architecture challenging the IoT heterogeneity. In 2016 3rd Smart Cloud Networks & Systems (SCNS), pp. 1–3: IEEE.

Download references

Funding

This study was supported by the National Natural Science Foundation of China vide (Grant no. 61672080) and the National Aerospace Science Foundation of China vide (Grant no. 2016ZD51031). Additionally, this work was supported in part by the Fundamental Research Funds for Central Universities vide (Grants FRF-BD-18-016A, ZFYY41402020502, and JSZL2017601B005) as well as the Key Lab of Information Network Security of Ministry of Public Security (The Third Research Institute of Ministry of Public Security).

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Shicheng Zhu & Yang Xu

  2. School of Reliability and Systems Engineering, Beihang University, Beijing, 100083, China

    Shunkun Yang & Xiaodong Gou

  3. Key Lab of Information Network Security, Ministry of Public Security, Shanghai, 200031, China

    Tao Zhang

  4. Run Technologies Co., Ltd. Beijing, Beijing, 100192, China

    Yueliang Wan

  5. Beijing Engineering Research Center for Cyberspace Data Analysis and Applications, Beijing, 100083, China

    Yueliang Wan

Authors
  1. Shicheng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shunkun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaodong Gou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yueliang Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SY and TZ directed the research and did the final revisions. SZ and XG wrote the paper, YX and YW have been involved in the writing of the manuscript. All the authors approved the final version of the manuscript.

Corresponding authors

Correspondence to Shunkun Yang or Tao Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, S., Yang, S., Gou, X. et al. Survey of Testing Methods and Testbed Development Concerning Internet of Things. Wireless Pers Commun 123, 165–194 (2022). https://doi.org/10.1007/s11277-021-09124-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-09124-5

Keywords

Search

Navigation