Skip to main content
Springer Nature Link
Log in

Enhancing gas detection-based swarming through deep reinforcement learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Swarm-Intelligence (SI), the collective behavior of decentralized and self-organized system, is used to efficiently carry out practical missions in various environments. To guarantee the performance of swarm, it is highly important that each object operates as an individual system while the devices are organized as simple as possible. This paper proposes an efficient, scalable, and practical swarming system using gas detection device. Each object of the proposed system has multiple sensors and detects gas in real time. To let the objects move toward gas rich spot, we propose two approaches for system design, vector-sum based, and Reinforcement Learning (RL) based. We firstly introduce our deterministic vector-sum-based approach and address the RL-based approach to extend the applicability and flexibility of the system. Through system performance evaluation, we validated that each object with a simple device configuration performs its mission perfectly in various environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Abraham L, Biju S, Biju F, Jose J, Kalantri R, Rajguru S (2019) Swarm robotics in disaster management. In: 2019 International Conference on Innovative Sustainable Computational Technologies (CISCT). IEEE, pp 1–5

  2. Babaeizadeh M, Frosio I, Tyree S, Clemons J, Kautz J (2016) Reinforcement learning through asynchronous advantage actor-critic on a gpu. arXiv preprintarXiv:1611.06256

  3. Beni G (1988) The concept of cellular robotic system. In: Proceedings IEEE International Symposium on Intelligent Control 1988. IEEE, pp 57–62

  4. Beni G, Wang J (1993) Swarm intelligence in cellular robotic systems. In: Robots and Biological Systems: Towards a New Bionics? Springer, pp 703–712

  5. Brambilla M, Ferrante E, Birattari M, Dorigo M (2013) Swarm robotics: a review from the swarm engineering perspective. Swarm Intell 7(1):1–41

    Article  Google Scholar 

  6. Cabot A, Dieguez A, Romano-Rodrıguez A, Morante J, Barsan N (2001) Influence of the catalytic introduction procedure on the nano-sno2 gas sensor performances: where and how stay the catalytic atoms? Sensors Actuators B: Chem 79(2–3):98–106

    Article  Google Scholar 

  7. Ceylan H, Yasa IC, Kilic U, Hu W, Sitti M (2019) Translational prospects of untethered medical microrobots. Progr Biomed Eng 1(1):012002

    Article  Google Scholar 

  8. Clark D (1988) The design philosophy of the darpa internet protocols. In: Symposium Proceedings on Communications Architectures and Protocols, pp 106–114

  9. Dayan P (2002) Reinforcement learning. Stevens’ Handbook of Experimental Psychology

  10. Dickerson JP, Kagan V, Subrahmanian V (2014) Using sentiment to detect bots on twitter: Are humans more opinionated than bots? In: 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014). IEEE, pp 620–627

  11. Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern Part B (Cybern) 26(1):29–41

    Article  Google Scholar 

  12. Dossi N, Toniolo R, Pizzariello A, Carrilho E, Piccin E, Battiston S, Bontempelli G (2012) An electrochemical gas sensor based on paper supported room temperature ionic liquids. Lab Chip 12(1):153–158

    Article  Google Scholar 

  13. Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: MHS’95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science. IEEE, pp 39–43

  14. Ehang egret’s 1374 drones dancing over the city wall of xi’an, achieving a guinness world records title. http://www.ehang.com/news/365.html. Accessed 24 May 2019

  15. Fan J, Wang Z, Xie Y, Yang Z (2020) A theoretical analysis of deep q-learning. In: Learning for Dynamics and Control. PMLR, pp 486–489

  16. Gilpin K, Knaian A, Rus D (2010) Robot pebbles: one centimeter modules for programmable matter through self-disassembly. In: 2010 IEEE international Conference on Robotics and Automation. IEEE, pp 2485–2492

  17. Gu S, Holly E, Lillicrap T, Levine S (2016) Deep reinforcement learning for robotic manipulation. arXiv preprintarXiv:1610.00633, 1

  18. Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning. PMLR, pp 1861–1870

  19. Hörtner H, Gardiner M, Haring R, Lindinger C, Berger F (2012) Spaxels, pixels in space. In: Proceedings of the International Conference on Signal Processing and Multimedia Applications and Wireless Information Networks and Systems. pp 19–24

  20. Hwang W-J, Shin K-S, Roh J-H, Lee D-S, Choa S-H (2011) Development of micro-heaters with optimized temperature compensation design for gas sensors. Sensors 11(3):2580–2591

    Article  Google Scholar 

  21. Intel drone light shows. https://inteldronelightshows.com/. Accessed 11 July 2020

  22. Jung J, Yoo S, La WG, Lee DR, Bae M, Kim H (2018) Avss: airborne video surveillance system. Sensors 18(6):1939

    Article  Google Scholar 

  23. Kennedy J (2006) Swarm intelligence. In: Handbook of Nature-Inspired and Innovative Computing. Springer, pp 187–219

  24. Larochelle H, Bengio Y, Louradour J, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10(1)

  25. Levin E, Pieraccini R, Eckert W (1998) Using markov decision process for learning dialogue strategies. In: Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98 (Cat. No. 98CH36181), vol 1. IEEE, pp 201–204

  26. Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971

  27. Liu X, Cheng S, Liu H, Hu S, Zhang D, Ning H (2012) A survey on gas sensing technology. Sensors 12(7):9635–9665

    Article  Google Scholar 

  28. Mavrovouniotis M, Li C, Yang S (2017) A survey of swarm intelligence for dynamic optimization: algorithms and applications. Swarm Evol Comput 33:1–17

    Article  Google Scholar 

  29. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602

  30. Park S, Oh Y, Hong D (2017) Disaster response and recovery from the perspective of robotics. Int J Precis Eng Manuf 18(10):1475–1482

    Article  Google Scholar 

  31. Park S, Kim HT, Kim H (2020) Vmcs: elaborating apf-based swarm intelligence for mission-oriented multi-uv control. IEEE Access

  32. Plappert M, Houthooft R, Dhariwal P, Sidor S, Chen RY, Chen X, Asfour T, Abbeel P, Andrychowicz M (2017) Parameter space noise for exploration. arXiv preprint arXiv:1706.01905

  33. Qin C, Yan Q, He G (2019) Integrated energy systems planning with electricity, heat and gas using particle swarm optimization. Energy 188:116044

    Article  Google Scholar 

  34. Ricco A, Martin S, Zipperian T (1985) Surface acoustic wave gas sensor based on film conductivity changes. Sensors Actuators 8(4):319–333

    Article  Google Scholar 

  35. Rubenstein M, Shen W-M (2010) Automatic scalable size selection for the shape of a distributed robotic collective. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp 508–513

  36. Sakai G, Matsunaga N, Shimanoe K, Yamazoe N (2001) Theory of gas-diffusion controlled sensitivity for thin film semiconductor gas sensor. Sensors Actuators B: Chem 80(2):125–131

    Article  Google Scholar 

  37. Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

  38. Sutton RS, Barto AG (1999) Reinforcement learning. J Cogn Neurosci 11(1):126–134

    Article  Google Scholar 

  39. Thrun MC, Ultsch A (2021) Swarm intelligence for self-organized clustering. Artif Intell 290:103237

    Article  MathSciNet  Google Scholar 

  40. Tilley J (2017) Automation, robotics, and the factory of the future. McKinsey. https://www.mckinsey.com/business-functions/operations/our-insights/automation-robotics-and-the-factory-of-the-future

  41. Vieira LFM, Lee U, Gerla M (2010) Phero-trail: a bio-inspired location service for mobile underwater sensor networks. IEEE J Selected Areas Commun 28(4):553–563

    Article  Google Scholar 

  42. Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach learn 8(3–4):229–256

    MATH  Google Scholar 

Download references

Acknowledgements

This research was supported by the Human Resources Program in Energy Technology of the Korea Institute of Energy Technology Evaluation and Planning (KETEP) and the Ministry of Trade, Industry & Energy (MOTIE) of the Republic of Korea (No. 20204010600220) and the National Research Foundation of Korea funded by the Korean Government (grant 2020R1A2C1012389).

Author information

Author notes

  1. Sangmin Lee and Seongjoon Park contributed equally to this work.

Authors and Affiliations

  1. Korea University, Seoul, Republic of Korea

    Sangmin Lee, Seongjoon Park & Hwangnam Kim

Authors
  1. Sangmin Lee
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Seongjoon Park
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Hwangnam Kim
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Hwangnam Kim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Preliminary version of this paper appeared in the Proceedings of the 6th International Conference on Next Generation Computing 2020 (ICNGC).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, S., Park, S. & Kim, H. Enhancing gas detection-based swarming through deep reinforcement learning. J Supercomput 78, 14794–14812 (2022). https://doi.org/10.1007/s11227-022-04478-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04478-4

Keywords