Skip to main content

Advertisement

SpringerLink
Log in

iCoCoA: intelligent congestion control algorithm for CoAP using deep reinforcement learning

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The constrained application protocol (CoAP) is an application layer protocol in IoT, with underlying support for congestion control mechanism. It minimizes the frequent retransmissions, but does not optimize the throughput or adapt dynamic conditions. However, designing an efficient congestion control mechanism over the IoT poses new challenges because of its resource constraint nature. In this context, this article presents a new Intelligent congestion control algorithm (iCoCoA) for constraint devices, motivated by the success of the deep reinforcement learning in various applications. The iCoCoA learns from the various network features to decide the best Retransmission Timeout to mitigate the congestion in the dynamic environments. It also optimizes the throughput, energy, and unnecessary frequent retransmissions compared with the existing models. iCoCoA is developed and tested on the Cooja simulator and compared it with the standard protocols such as CoAP, CoCoA, and CoCoA+ in continuous and burst traffic conditions. The proposed iCoCoA mitigates congestion, outperforms 4–15% in throughput, 3–10% better packet delivery ratio, and 7–16% energy-efficiency with reduced number of retransmissions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability statement

All data generated or analysed during this study are generated randomly during the simulation. The details about data generation is included in this article.

References

  • Agyemang JO, Kponyo JJ, Gadze JD, Nunoo-Mensah H, Yu D (2022) Lightweight messaging protocol for internet of things devices. Technologies 10(1):21

  • Aimtongkham P, Horkaew P, So-In C (2021) An enhanced CoAP scheme using fuzzy logic with adaptive timeout for IoT congestion control. IEEE Access 9:58967–58981

    Article  Google Scholar 

  • Akpakwu GA, Hancke GP, Abu-Mahfouz AM (2020) CACC: context-aware congestion control approach for lightweight CoAP/UDP-based internet of things traffic. Trans Emerg Telecommun Technol 31(2):e3822

    Google Scholar 

  • Betzler A, Gomez C, Demirkol I, Paradells J (2013) Congestion control in reliable CoAP communication. In: Proceedings of the 16th ACM International Conference on Modeling, analysis & simulation of wireless and mobile systems. ACM, pp 365–372

  • Betzler A, Gomez C, Demirkol I, Paradells J (2015) CoCoA+: an advanced congestion control mechanism for CoAP. Ad Hoc Netw 33:126–139

    Article  Google Scholar 

  • Betzler A, Gomez C, Demirkol I, Paradells J (2016a) CoAP congestion control for the Internet of Things. IEEE Commun Mag 54(7):154–160

    Article  Google Scholar 

  • Betzler A, Isern J, Gomez C, Demirkol I, Paradells J (2016b) Experimental evaluation of congestion control for CoAP communications without end-to-end reliability. Ad Hoc Nets 52:183–194

    Article  Google Scholar 

  • Bolettieri S, Tanganelli G, Vallati C, Mingozzi E (2018) pCoCoA: a precise congestion control algorithm for CoAP. Ad Hoc Netw 80:116–129

    Article  Google Scholar 

  • Bormann C, Castellani AP, Shelby Z (2012) CoAP: an application protocol for billions of tiny internet nodes. IEEE Internet Comput 2:62–67

    Article  Google Scholar 

  • Demir AK, Abut F (2020) mlCoCoA: a machine learning-based congestion control for CoAP. Turk J Electr Eng Comput sci 28(5):1–20

    Google Scholar 

  • Donta PK, Amgoth T, Annavarapu CSR (2020) Congestion-aware data acquisition with q-learning for wireless sensor networks. In: 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS). IEEE, pp 1–6

  • Donta PK, Amgoth T, Annavarapu CSR (2021) An extended aco-based mobile sink path determination in wireless sensor networks. J Ambient Intell Humaniz Comput 12(10):8991–9006

    Article  Google Scholar 

  • Donta PK, Srirama SN, Amgoth T, Annavarapu CSR (2022) Survey on recent advances in iot application layer protocols and machine learning scope for research directions. Digit Commun Netw 8(5):727–744

    Article  Google Scholar 

  • HoBfeld T, Skorin-Kapov L, Heegaard PE, Varela M (2017) Definition of QoE fairness in shared systems. IEEE Commun Lett 21(1):184–187

    Article  Google Scholar 

  • Jamshed MA, Ali K, Abbasi QH, Imran MA, Ur-Rehman M (2022) Challenges, applications and future of wireless sensors in internet of things: a review. IEEE Sens J 22(6):5482–5494

    Article  Google Scholar 

  • Jay N, Rotman N, Godfrey B, Schapira M, Tamar A (2019) A deep reinforcement learning perspective on internet congestion control. In: International Conference on machine learning, pp 3050–3059

  • Kaur N, Sood SK (2017) An energy-efficient architecture for the internet of things. IEEE Syst J 11(2):796–805

    Article  Google Scholar 

  • Kim M, Lee S, Khan MTR, Seo J, Bae Y, Jeong Y, Kim D (2019) A new CoAP congestion control scheme using message loss feedback for IoUT. In: Proceedings of the 34th ACM/SIGAPP Symposium on applied computing. SAC ’19. ACM, pp 2385–2390

  • Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  • Lippmann R (1987) An introduction to computing with neural nets. IEEE ASSP Mag 4(2):4–22

    Article  Google Scholar 

  • Mahajan N, Chauhan A, Kumar H, Kaushal S, Sangaiah AK (2022) A deep learning approach to detection and mitigation of distributed denial of service attacks in high availability intelligent transport systems. Mob Netw Appl 20:1–21

    Google Scholar 

  • Martinez B, Monton M, Vilajosana I, Prades JD (2015) The power of models: Modeling power consumption for IoT devices. IEEE Sens J 15(10):5777–5789

    Article  Google Scholar 

  • Mišić J, Ali MZ, Mišić VB (2018) Architecture for IoT domain with CoAP observe feature. IEEE Internet Things J 5(2):1196–1205

    Article  Google Scholar 

  • Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529

    Article  Google Scholar 

  • Nie X, Zhao Y, Li Z, Chen G, Sui K, Zhang J, Ye Z, Pei D (2019) Dynamic TCP initial windows and congestion control schemes through reinforcement learning. IEEE J Sel Areas Commun 37(6):1231–1247

    Article  Google Scholar 

  • Praveen Kumar D, Tarachand A, Rao ACS (2019) Machine learning algorithms for wireless sensor networks: a survey. Inf Fusion 49:1–25

    Article  Google Scholar 

  • Rathod V, Jeppu N, Sastry S, Singala S, Tahiliani MP (2019) CoCoA++: delay gradient based congestion control for Internet of Things. Future Gener Comput Syst 100:1053–1072

    Article  Google Scholar 

  • Salkuti SR (2018) Congestion management using optimal transmission switching. IEEE Syst J 12(4):3555–3564

    Article  Google Scholar 

  • Sandell M, Raza U (2019) Application layer coding for IoT: benefits, limitations, and implementation aspects. IEEE Syst J 13(1):554–561

    Article  Google Scholar 

  • Sangaiah AK, Ramamoorthi JS, Rodrigues JJ, Rahman MA, Muhammad G, Alrashoud M (2020) LACCVoV: linear adaptive congestion control with optimization of data dissemination model in vehicle-to-vehicle communication. IEEE Trans Intell Transp Syst 22(8):5319–5328

    Article  Google Scholar 

  • Sargent M, Allman M, Paxson V (2011) Computing TCP’s retransmission timer. Computing

  • Sun X, Ansari N (2018) Traffic load balancing among brokers at the IoT application layer. IEEE Trans Netw Serv Manag 15(1):489–502

    Article  Google Scholar 

  • Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press

    MATH  Google Scholar 

  • Suwannapong C, Khunboa C (2019) Congestion control in CoAP observe group communication. Sensors 19(15):3433

    Article  Google Scholar 

  • Suwannapong C, Khunboa C (2021) EnCoCo-RED: enhanced congestion control mechanism for CoAP observe group communication. Ad Hoc Netw 112:102377

    Article  Google Scholar 

  • Uroz D, Rodríguez RJ (2022) Characterization and evaluation of IoT protocols for data exfiltration. IEEE Internet of Things J 9(19):19062–19072

    Article  Google Scholar 

  • Xiao K, Mao S, Tugnait JK (2019) TCP-Drinc: smart congestion control based on deep reinforcement learning. IEEE Access 7:11892–11904

    Article  Google Scholar 

  • Yadav RK, Singh N, Piyush P (2020) Genetic CoCoA++: genetic algorithm based congestion control in CoAP. In: 2020 4th International Conference on intelligent computing and control systems (ICICCS). IEEE, pp 808–813

  • Zhang S, You X, Zhang P, Huang M, Li S (2022) A UCB-based dynamic CoAP mode selection algorithm in distribution IoT. Alex Eng J 61(1):719–727

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Archimedes Foundation under the Dora plus Grant 11-15/OO/11476 and SERB, India, through grant CRG/2021/003888. We also thank financial support to UoH-IoE by MHRD, India (F11/9/2019-U3(A)).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, Jharkhand, India

    Praveen Kumar Donta, Tarachand Amgoth & Chandra Sekhara Rao Annavarapu

  2. Mobile & Cloud Lab, Institute of Computer Science, University of Tartu, 50090, Tartu, Estonia

    Praveen Kumar Donta & Satish Narayana Srirama

  3. School of Computer and Information Sciences, University of Hyderabad, Hyderabad, 500 046, India

    Satish Narayana Srirama

Authors
  1. Praveen Kumar Donta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satish Narayana Srirama
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tarachand Amgoth
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chandra Sekhara Rao Annavarapu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Satish Narayana Srirama.

Ethics declarations

Conflict of interest

There are no potential conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Donta, P.K., Srirama, S.N., Amgoth, T. et al. iCoCoA: intelligent congestion control algorithm for CoAP using deep reinforcement learning. J Ambient Intell Human Comput 14, 2951–2966 (2023). https://doi.org/10.1007/s12652-023-04534-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-023-04534-8

Keywords

Search

Navigation