skip to main content
10.1145/3380446.3430619acmconferencesArticle/Chapter ViewAbstractPublication PagesmlcadConference Proceedingsconference-collections
research-article

Cost Optimization at Early Stages of Design Using Deep Reinforcement Learning

Authors Info & Claims
Published:16 November 2020Publication History

ABSTRACT

With the increase in the complexity of the modern system on Chips(SoCs) and the demand for a lower time-to-market, automation becomes essential in hardware design. This is particularly relevant in complex/time-consuming tasks, as the optimization of design cost for a hardware component. Design cost, in fact, may depend on several objectives, as for the hardware-software trade-off. Given the complexity of this task, the designer often has no means to perform a fast and effective optimization in particular for larger and complex designs. In this paper, we introduce Deep Reinforcement Learning(DRL) for design cost optimization at the early stages of the design process. We first show that DRL is a perfectly suitable solution for the problem at hand. Afterward, by means of a Pointer Network, a neural network specifically applied for combinatorial problems, we benchmark three DRL algorithms towards the selected problem. Results obtained in different settings show the improvements achieved by DRL algorithms compared to conventional optimization methods. Additionally, by using reward redistribution proposed in the recently introduced RUDDER method, we obtain significant improvements in complex designs. Here, the obtained optimization is on average 15.18% on the area as well as 8.25% and 8.12% on the application size and execution time on a dataset of industrial hardware/software interface design

Skip Supplemental Material Section

Supplemental Material

3380446.3430619.mp4

With the increase in the complexity of the modern System on Chips (SoCs) and the demand for a lower time-to-market, automation becomes essential in hardware design. This is particularly relevant in complex/time-consuming tasks, as the optimization of design cost for a hardware component. Design cost, in fact, may depend on several objectives, as for the hardware-software trade-off. Given the complexity of this task, the designer often has no means to perform a fast and effective optimization?in particular for larger and complex designs. In this paper, we introduce Deep Reinforcement Learning (DRL) for design cost optimization at the early stages of the design process. We first show that DRL is a perfectly suitable solution for the problem at hand. Afterward, through the use of a Pointer Network, a neural network specifically applied for combinatorial problems, we benchmark five DRL algorithms towards the selected problem. Results obtained in different settings show the improvements achieved by DRL algorithms.

References

  1. E. J Anderson et al. 1994. Genetic algorithms for combinatorial optimization: the assemble line balancing problem. ORSA Journal on Computing (1994).Google ScholarGoogle Scholar
  2. J. A. Arjona-Medina et al. 2019. RUDDER: Return decomposition for delayed rewards. In NeurIPS.Google ScholarGoogle Scholar
  3. Irwan Bello et al. 2016. Neural Combinatorial Optimization with Reinforcement Learning. (2016).Google ScholarGoogle Scholar
  4. A. Colorni et al. 1996. Heuristics from nature for hard combinatorial optimization problems. International Transactions in Operational Research (1996).Google ScholarGoogle Scholar
  5. W. Ecker et al. 2017. Metamodeling and code generation in the hardware/software interface domain. In Handbook of Hard./Soft. Codesign.Google ScholarGoogle Scholar
  6. W. Ecker and others. 2009. Hardware-dependent Software: Principles and Practice. Springer Publishing Company, Incorporated. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. I. Goodfellow et al. 2016. Deep Learning. MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A. Graves et al. 2005. Bidirectional LSTM networks for improved phoneme classification and recognition. In ICANN. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. H. Hu et al. 2017. Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method. (2017).Google ScholarGoogle Scholar
  10. B. Korte et al. 2012. Combinatorial optimization. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. A. Laterre et al. 2018. Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization. (2018).Google ScholarGoogle Scholar
  12. A. Mirhoseini et al. 2020. Chip Placement with Deep Reinforcement Learning. arXiv preprint arXiv:2004.10746 (2020).Google ScholarGoogle Scholar
  13. J. Schulman et al. 2017. Proximal Policy Optimization Algorithms.Google ScholarGoogle Scholar
  14. L. Servadei et al. 2019. Accurate Cost Estimation of Memory Systems Inspired by Machine Learning for Computer Vision. In Design, Automation Test in Europe Conf. Exh. (DATE).Google ScholarGoogle Scholar
  15. F. Streit et al. 2018. Model-based design automation of hardware/software codesigns for Xilinx Zynq PSoCs. In 2018 International Conference on ReConFigurable Computing and FPGAs (ReConFig).Google ScholarGoogle ScholarCross RefCross Ref
  16. R. Sutton et al. 2018. Reinforcement Learning: An Introduction. A Bradford Book. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. O. Vinyals et al. 2015. Pointer networks. In NIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Cost Optimization at Early Stages of Design Using Deep Reinforcement Learning

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      MLCAD '20: Proceedings of the 2020 ACM/IEEE Workshop on Machine Learning for CAD
      November 2020
      183 pages
      ISBN:9781450375191
      DOI:10.1145/3380446

      Copyright © 2020 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 16 November 2020

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Upcoming Conference

      MLCAD '24
      2024 ACM/IEEE International Symposium on Machine Learning for CAD
      September 9 - 11, 2024
      Salt Lake City , UT , USA

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader