Skip to main content

Advertisement

SpringerLink
Log in

Reliability-aware intelligent mapping based on reinforcement learning for networks-on-chips

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Designing reliable Networks on Chip (NoCs) is critical, especially with the continuous scaling down of integrated circuit technology, which exposes NoCs to various types of faults. In this paper, a reliability-aware application mapping technique for improving the reliability of heterogeneous NoCs is proposed. It is based on a hybridization of the Multi-Objective Particle Swarp Optimization (MOPSO) algorithm and Reinforcement Learning (RL). At design time, MOPSO and RL perform the optimization of an initial mapping and the prediction of fault-tolerant remapping scenarios, respectively. To teach and to produce an intelligent agent capable of generating an optimal adaptive remapping scheme to address run-time permanent processing element (PE) failures. Two models of RL agents are trained, each based on a different mechanism of task migration: 1) step-based agent and 2) swap-based agent. Experiments were carried out to assess the performance of our innovative technique on various sizes of NoCs, using real benchmarks and varying the levels of heterogeneity and failure in the NoC. The results of the experiments reveal that using RL to solve the reliability problem in NoCs yields interesting results in terms of reliability, energy consumption, execution time, and cost migration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. Intellectual Property.

References

  1. Dally WJ, Towles B (2001) Route packets, not wires: On-chip interconnection networks. In: Design Automation Conference, 2001. Proceedings, pp. 684–689. https://doi.org/10.1109/DAC.2001.156225. IEEE

  2. Sgroi M, Sheets M, Mihal A, Keutzer K, Malik S, Rabaey J, Sangiovanni-Vincentelli A (2001) Addressing the system-on-a-chip interconnect woes through communication-based design. In: Proceedings of the 38th Design Automation Conference (IEEE Cat. No. 01CH37232), pp. 667–672. https://doi.org/10.1145/378239.379045. IEEE

  3. Benini L, De Micheli G (2002) Networks on chips: a new soc paradigm. Computer 35(1):70–78. https://doi.org/10.1109/2.976921

    Article  Google Scholar 

  4. Kumar S, Jantsch A, Soininen J-P, Forsell M, Millberg M, Oberg J, Tiensyrja K, Hemani A (2002) A network on chip architecture and design methodology. In: VLSI, 2002. Proceedings. IEEE Computer Society Annual Symposium On, pp. 117–124. https://doi.org/10.1109/ISVLSI.2002.1016885. IEEE

  5. Radetzki M, Feng C, Zhao X, Jantsch A (2013) Methods for fault tolerance in networks-on-chip. ACM Comput Surv (CSUR) 46(1):8. https://doi.org/10.1145/2522968.2522976

    Article  Google Scholar 

  6. Constantinescu C (2003) Trends and challenges in VLSI circuit reliability. IEEE Micro 23(4):14–19. https://doi.org/10.1109/MM.2003.1225959

    Article  Google Scholar 

  7. Chatterjee N, Mukherjee P, Chattopadhyay S (2018) Reliability-aware application mapping onto mesh based network-on-chip. Integration 62:92–113. https://doi.org/10.1016/j.vlsi.2018.02.002

    Article  Google Scholar 

  8. Gupta M, Bhargava L, Indu S (2021) Mapping techniques in multicore processors: current and future trends. J Supercomput 87:1–56. https://doi.org/10.1007/s11227-021-03650-6

    Article  Google Scholar 

  9. Kadri N, Koudil M (2019) A survey on fault-tolerant application mapping techniques for network-on-chip. J Syst Arch 92:39–52. https://doi.org/10.1016/j.sysarc.2018.10.001

    Article  Google Scholar 

  10. Sahu PK, Chattopadhyay S (2013) A survey on application mapping strategies for network-on-chip design. J Syst Arch 59(1):60–76. https://doi.org/10.1016/j.sysarc.2012.10.004

    Article  Google Scholar 

  11. Abualigah L, Yousri D, Abd Elaziz M, Ewees AA, Al-Qaness MA, Gandomi AH (2021) Aquila optimizer: a novel meta-heuristic optimization algorithm. Computers Industrial Eng 157:107250. https://doi.org/10.1016/j.cie.2021.107250

    Article  Google Scholar 

  12. Abualigah L, Abd Elaziz M, Sumari P, Geem ZW, Gandomi AH (2022) Reptile search algorithm (rsa): a nature-inspired meta-heuristic optimizer. Expert Syst Appl 191:116158. https://doi.org/10.1016/j.eswa.2021.116158

    Article  Google Scholar 

  13. Oyelade ON, Ezugwu AE-S, Mohamed TI, Abualigah L (2022) Ebola optimization search algorithm: a new nature-inspired metaheuristic optimization algorithm. IEEE Access 10:16150–16177. https://doi.org/10.1109/ACCESS.2022.3147821

    Article  Google Scholar 

  14. Agushaka JO, Ezugwu AE, Abualigah L (2022) Dwarf mongoose optimization algorithm. Computer Methods Appl Mech Eng 391:114570. https://doi.org/10.1016/j.cma.2022.114570

    Article  MathSciNet  MATH  Google Scholar 

  15. Abualigah L, Diabat A, Mirjalili S, Abd Elaziz M, Gandomi AH (2021) The arithmetic optimization algorithm. Computer Methods Appl Mech Eng 376:113609. https://doi.org/10.1016/j.cma.2020.113609

    Article  MathSciNet  MATH  Google Scholar 

  16. Kalahroudi PM, Yaghoubi E, Barekatain B (2021) Iam: an improved mapping on a 2-d network on chip to reduce communication cost and energy consumption. Photonic Netw Commun 41(1):78–92. https://doi.org/10.1007/s11107-020-00911-x

    Article  Google Scholar 

  17. Paul S, Chatterjee N, Ghosal P (2021) Dynamic task allocation and scheduling with contention-awareness for network-on-chip based multicore systems. J Syst Arch 115:102020

    Article  Google Scholar 

  18. Das A, Kumar A, Veeravalli B (2014) Communication and migration energy aware task mapping for reliable multiprocessor systems. Fut Gener Computer Syst 30:216–228. https://doi.org/10.1016/j.future.2013.06.016

    Article  Google Scholar 

  19. Das A, Kumar A, Veeravalli B (2012) Energy-aware communication and remapping of tasks for reliable multimedia multiprocessor systems. In: Parallel and Distributed Systems (ICPADS), 2012 IEEE 18th International Conference On, pp. 564–571. https://doi.org/10.1109/ICPADS.2012.82. IEEE

  20. Das A, Kumar A (2012) Fault-aware task re-mapping for throughput constrained multimedia applications on noc-based mpsocs. In: Rapid System Prototyping (RSP), 2012 23rd IEEE International Symposium On, pp. 149–155. https://doi.org/10.1109/RSP.2012.6380704. IEEE

  21. Das A, Kumar A, Veeravalli B (2013) Communication and migration energy aware design space exploration for multicore systems with intermittent faults. In: Proceedings of the Conference on Design, Automation and Test in Europe, pp. 1631–1636. https://doi.org/10.7873/DATE.2013.331. EDA Consortium

  22. Das A, Singh AK, Kumar A (2013) Energy-aware dynamic reconfiguration of communication-centric applications for reliable mpsocs. In: reconfigurable and communication-centric systems-on-chip (ReCoSoC), 2013 8th International Workshop On, pp. 1–7. https://doi.org/10.1109/ReCoSoC.2013.6581540. IEEE

  23. Derin O, Kabakci D, Fiorin L (2011) Online task remapping strategies for fault-tolerant network-on-chip multiprocessors. In: Proceedings of the Fifth ACM/IEEE International Symposium on Networks-on-Chip, pp. 129–136. https://doi.org/10.1145/1999946.1999967. ACM

  24. Derin O, Fiorin L (2014) Towards a reliability-aware design flow for kahn process networks on noc-based multiprocessors. In: architecture of computing systems (ARCS), 2014 Workshop Proceedings, pp. 1–8. VDE

  25. Ababei C, Katti R (2009) Achieving network on chip fault tolerance by adaptive remapping. In: Parallel & Distributed Processing, 2009, IEEE International Symposium, vol. 4. https://doi.org/10.1109/IPDPS.2009.5161202

  26. Chatterjee N, Paul S, Chattopadhyay S (2017) Fault-tolerant dynamic task mapping and scheduling for network-on-chip-based multicore platform. ACM Trans Embedded Comput Syst (TECS) 16(4):108. https://doi.org/10.1145/3055512

    Article  Google Scholar 

  27. Bhanu PV, Kulkarni PV, Avadhanam, SP, Soumya J, Cenkeramaddi LR (2019) Multi-application based fault-tolerant network-on-chip design for mesh topology using reconfigurable architecture. In: International Symposium on VLSI Design and Test, pp. 442–454. https://doi.org/10.1007/978-981-32-9767-8_37. Springer

  28. Bolchini C, Carminati M, Miele A, Das A Kumar A, Veeravalli B (2013) Run-time mapping for reliable many-cores based on energy/performance trade-offs. In: defect and fault tolerance in vlsi and nanotechnology systems (DFT), 2013 IEEE International Symposium On, pp. 58–64. https://doi.org/10.1109/DFT.2013.6653583. IEEE

  29. Das A, Kumar A, Veeravalli B (2014) Energy-aware task mapping and scheduling for reliable embedded computing systems. ACM Trans Embedded Comput Syst (TECS) 13(2s):72. https://doi.org/10.1145/2544375.2544392

    Article  Google Scholar 

  30. Namazi A, Abdollahi M, Safari S, Mohammadi S, Daneshtalab M (2018) Lrtm: Life-time and reliability-aware task mapping approach for heterogeneous multi-core systems. In: 2018 11th International Workshop on Network on Chip Architectures (NoCArc), pp. 1–6. https://doi.org/10.1109/NOCARC.2018.8541223. IEEE

  31. Chou C-L, Marculescu R (2011) Farm: Fault-aware resource management in noc-based multiprocessor platforms. In: 2011 Design, Automation & Test in Europe, pp. 1–6. https://doi.org/10.1109/DATE.2011.5763113. IEEE

  32. Beechu NKR, Harishchandra VM, Balachandra NKY (2017) High-performance and energy-efficient fault-tolerance core mapping in noc. Sustain Comput : Inf Syst 16:1–10. https://doi.org/10.1016/j.suscom.2017.08.004

    Article  Google Scholar 

  33. Bhanu PV, Kulkarni P, Soumya J, Cenkarmaddi LR, Idsøe H (2018) Torus topology based fault-tolerant network-on-chip design with flexible spare core placement. In: 2018 14th Conference on Ph. D. Research in Microelectronics and Electronics (PRIME), pp. 97–100. https://doi.org/10.1109/PRIME.2018.8430343. IEEE

  34. Bhanu PV, Kulkarni PV (2019) Fault-tolerant network-on-chip design with flexible spare core placement. ACM J Emerg Technol Comput Syst (JETC) 15(1):5. https://doi.org/10.1145/3269983

    Article  Google Scholar 

  35. Bhanu PV, Soumya J (2021) Fault-tolerant application mapping on mesh-of-tree based network-on-chip. J Syst Arch 116:102026. https://doi.org/10.1016/j.sysarc.2021.102026

    Article  Google Scholar 

  36. Wu N, Xie Y (2022) A survey of machine learning for computer architecture and systems. ACM Comput Surv (CSUR) 55(3):1–39. https://doi.org/10.1145/3494523

    Article  Google Scholar 

  37. Wang K, Louri A, Karanth A, Bunescu R (2019) High-performance, energy-efficient, fault-tolerant network-on-chip design using reinforcement learning. In: 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1166–1171. https://doi.org/10.23919/DATE.2019.8714869. IEEE

  38. Wang K, Louri A, Karanth A, Bunescu R (2019) Intellinoc: A holistic design framework for energy-efficient and reliable on-chip communication for manycores. In: 2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA), pp. 1–12. https://doi.org/10.1145/3307650.3322274. IEEE

  39. Wang K, Louri A (2020) Cure: a high-performance, low-power, and reliable network-on-chip design using reinforcement learning. IEEE Trans Parallel Distrib Syst 31(9):2125–2138. https://doi.org/10.1109/TPDS.2020.2986297

    Article  Google Scholar 

  40. Samala J, Takawale H, Chokhani Y, Bhanu PV, Soumya J (2020) Fault-tolerant routing algorithm for mesh based noc using reinforcement learning. In: 2020 24th International Symposium on VLSI Design and Test (VDAT), pp. 1–6. https://doi.org/10.1109/VDAT50263.2020.9190340. IEEE

  41. Jagadheesh S, Bhanu PV, Soumya J, Cenkeramaddi LR (2022) Reinforcement learning based fault-tolerant routing algorithm for mesh based noc and its fpga implementation. IEEE Access. https://doi.org/10.1109/ACCESS.2022.3168992

  42. Amin W, Hussain F, Anjum S, Khan S, Baloch NK, Nain Z, Kim SW (2020) Performance evaluation of application mapping approaches for network-on-chip designs. IEEE Access 8:63607–63631. https://doi.org/10.1109/ACCESS.2020.2982675

    Article  Google Scholar 

  43. Chen Q, Huang W, Zhang Y, Huang Y (2020) An ip core mapping algorithm based on neural networks. IEEE Trans Very Large Scale Integr (VLSI) Syst 29(1):189–202. https://doi.org/10.1109/TVLSI.2020.3033658

    Article  Google Scholar 

  44. Chen Q, Huang W, Peng Y, Huang Y (2021) A reinforcement learning-based framework for solving the ip mapping problem. IEEE Trans Very Large Scale Integr (VLSI) Syst 29(9):1638–1651. https://doi.org/10.1109/TVLSI.2021.3097712

    Article  Google Scholar 

  45. Kadri N, Koudil M (2022) Multi-objective biogeography-based optimization and reinforcement learning hybridization for network-on chip reliability improvement. J Parallel Distrib Comput 161:20–36. https://doi.org/10.1016/j.jpdc.2021.11.005

    Article  Google Scholar 

  46. Darbandi M, Ramtin AR, Sharafi OK (2020) Tasks mapping in the network on a chip using an improved optimization algorithm. Int J Pervasive Comput Communi. https://doi.org/10.1108/IJPCC-07-2019-0053

    Article  Google Scholar 

  47. Lei, W., Xiang, L.: Energy-and latency-aware noc mapping based on chaos discrete particle swarm optimization. In: 2010 International Conference on Communications and Mobile Computing, vol. 1, pp. 263–268 (2010). https://doi.org/10.1109/CMC.2010.38. IEEE

  48. Fang J, Zong H, Zhao H, Cai H (2019) Intelligent mapping method for power consumption and delay optimization based on heterogeneous noc platform. Electronics 8(8):912. https://doi.org/10.3390/electronics8080912

    Article  Google Scholar 

  49. Hu J, Marculescu R (2005) Energy-and performance-aware mapping for regular noc architectures. IEEE Trans Computer-aided Des Integr Circuits Syst 24(4):551–562. https://doi.org/10.1109/TCAD.2005.844106

    Article  Google Scholar 

  50. Bonney C, Campos P, Dahir N, Tempesti G (2016) Fault tolerant task mapping on many-core arrays. In: Computational Intelligence (SSCI), 2016 IEEE Symposium Series On, pp. 1–8. https://doi.org/10.1109/SSCI.2016.7850174. IEEE

  51. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-International Conference on Neural Networks, vol. 4, pp. 1942–1948. https://doi.org/10.1109/ICNN.1995.488968. IEEE

  52. Sahu PK, Shah T, Manna K, Chattopadhyay S (2013) Application mapping onto mesh-based network-on-chip using discrete particle swarm optimization. IEEE Trans Very Large Scale Integr (VLSI) Syst 22(2):300–312. https://doi.org/10.1109/TVLSI.2013.2240708

    Article  Google Scholar 

  53. Luo G, Zhao H, Song C (2008) Convergence analysis of a dynamic discrete pso algorithm. In: 2008 First International Conference on Intelligent Networks and Intelligent Systems, pp. 89–92. https://doi.org/10.1109/ICINIS.2008.100. IEEE

  54. Alvarez-Benitez JE, Everson RM, Fieldsend JE (2005) A mopso algorithm based exclusively on pareto dominance concepts. In: International Conference on Evolutionary Multi-criterion Optimization, pp. 459–473. https://doi.org/10.1007/978-3-540-31880-4_32. Springer

  55. Arora JS (2017) Chapter 18 – multi-objective optimum design concepts and methods, pp. 558–559. https://doi.org/10.1016/B978-0-12-800806-5.00018-4

  56. Bolanda N, Charkhgard H, Savelsbergha M (2016) On the existence of ideal solutions in multi-objective 0-1 integer programs. Technical report, Working Paper

  57. Jang B, Kim M, Harerimana G, Kim JW (2019) Q-learning algorithms: a comprehensive classification and applications. IEEE Access 7:133653–133667. https://doi.org/10.1109/ACCESS.2019.2941229

    Article  Google Scholar 

  58. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Magaz 34(6):26–38. https://doi.org/10.1109/msp.2017.2743240

    Article  Google Scholar 

  59. Hasselt HV (2010) Double q-learning. In: Advances in Neural Information Processing Systems, pp. 2613–2621

  60. Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. In: Thirtieth AAAI Conference on Artificial Intelligence

  61. Van Der Tol EB, Jaspers EG (2001) Mapping of mpeg-4 decoding on a flexible architecture platform. In: Media Processors 2002, vol. 4674, pp. 1–13. https://doi.org/10.1117/12.451067. International Society for Optics and Photonics

  62. Bertozzi D, Jalabert A, Murali S, Tamhankar R, Stergiou S, Benini L, De Micheli G (2005) Noc synthesis flow for customized domain specific multiprocessor systems-on-chip. IEEE Trans Parallel Distrib Syst 16(2):113–129. https://doi.org/10.1109/TPDS.2005.22

    Article  Google Scholar 

  63. Kundu S, Chattopadhyay S (2014) Network-on-chip: the next generation of system-on-chip integration. CRC Press, Boston

    Google Scholar 

  64. Zhou W, Zhang Y, Mao Z (2006) Pareto based multi-objective mapping ip cores onto noc architectures. In: Circuits and Systems, 2006. APCCAS 2006. IEEE Asia Pacific Conference On, pp. 331–334. https://doi.org/10.1109/apccas.2006.342418. IEEE

  65. Le Q, Yang G, Hung WN, Zhang X, Fan F (2014) A multiobjective scatter search algorithm for fault-tolerant noc mapping optimisation. Int J Electron 101(8):1056–1073. https://doi.org/10.1080/00207217.2013.805392

    Article  Google Scholar 

  66. Murali S, Benini L, De Micheli G (2005) Mapping and physical planning of networks-on-chip architectures with quality-of-service guarantees. In: Proceedings of the 2005 Asia and South Pacific Design Automation Conference, pp. 27–32. https://doi.org/10.1109/ASPDAC.2005.1466124

  67. Mahabadi A, Zahedi S, Khonsari A (2013) Reliable energy-aware application mapping and voltage-frequency island partitioning for gals-based noc. J Computer Syst Sci 79(4):457–474. https://doi.org/10.1016/j.jcss.2012.09.006

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire des Méthodes de Conception des Systèmes (LMCS), Ecole nationale Supérieure d’Informatique (ESI), BP 68M, 16270, Oued-Smar, Alger, Algeria

    Nassima Kadri, Azzeddine Chenine, Zakaria Laib & Mouloud Koudil

Authors
  1. Nassima Kadri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Azzeddine Chenine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zakaria Laib
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mouloud Koudil
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nassima Kadri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kadri, N., Chenine, A., Laib, Z. et al. Reliability-aware intelligent mapping based on reinforcement learning for networks-on-chips. J Supercomput 78, 18153–18188 (2022). https://doi.org/10.1007/s11227-022-04590-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04590-5

Keywords

Search

Navigation