Skip to main content
SpringerLink
Log in

GEMBench: A Platform for Collaborative Development of GPU Accelerated Embedded Markov Decision Systems

  • Conference paper
  • First Online:
Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11733))

Included in the following conference series:

  • 1433 Accesses

Abstract

Markov Decision Processes (MDPs) provide a powerful decision making framework, which is increasingly being used in the design of Embedded Computing Systems (ECSs). This paper presents a detailed accounting of the use of MDPs in this context across research groups, including reference implementations, common datasets, file formats and platforms. Inspired by recent results showing the promising outlook of using embedded GPUs to solve MDPs on ECSs, we detail the many challenges that designers currently face and present GEMBench (the Gpu accelerated Embedded Mdp testBench) in order to facilitate experimental research in this area. GEMBench is targeted to a specific embedded GPU platform, the NVIDIA Jetson platform, and is designed for future retargetability to other platforms. GEMBench is a novel open source software package that is intended to run on the target platform. The package contains libraries of MDP solvers, parsers, datasets and reference solutions, which provide a comprehensive infrastructure for understanding trade-offs among existing embedded MDP techniques, and experimenting with novel techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. GEMBench Package (2019). https://ece.umd.edu/DSPCAD/projects/csm/packages/gembench.tar.gz. Accessed 14 Mar 2019

  2. Benini, L., Bogliolo, A., Paleologo, G.A., De Micheli, G.: Policy optimization for dynamic power management. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 18(6), 742–760 (1999)

    Article  Google Scholar 

  3. Brock, O., Trinkle, J., Ramos, F.: SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces. MIT Press, Cambridge (2009)

    Book  Google Scholar 

  4. Cassandra, A.R.: Exact and approximate algorithms for partially observable Markov decision processes. Ph.D. thesis, Brown University (1998)

    Google Scholar 

  5. Debizet, Y., Lallement, G., Abouzeid, F., Roche, P., Autran, J.: Q-learning-based adaptive power management for IoT system-on-chips with embedded power states. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5, May 2018

    Google Scholar 

  6. Elod, P.: Vision-based quadcopter navigation for following indoor corridors and outdoor railways. Master’s thesis, Technical University of Cluj-Napoca (2014)

    Google Scholar 

  7. Fackler, P.L.: MDPSOLVE: A MATLAB toolbox for solving Markov decision problems with dynamic programming – user’s guide. North Carolina State University. Technical report, January 2011

    Google Scholar 

  8. Hoey, J., St-Aubin, R., Hu, A., Boutilier, C.: SPUDD: stochastic planning using decision diagrams. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 279–288 (1999)

    Google Scholar 

  9. Jung, E., Maker, F., Cheung, T.L., Liu, X., Akella, V.: Markov decision process (MDP) framework for software power optimization using call profiles on mobile phones. J. Des. Autom. Embedded Syst. 14(2), 131–159 (2010)

    Article  Google Scholar 

  10. Maniloff, D.: Libpomdp. https://www.cs.uic.edu/~dmanilof/code.html. Accessed 05 Jan 2019

  11. McDermott, D.: PDDL — The Planning Domain Definition Language. Yale University, New Haven (1998)

    Google Scholar 

  12. Murphy, K.: Markov decision process toolbox for MATLAB. https://www.cs.ubc.ca/~murphyk/Software/MDP/mdp.html. Accessed 05 Jan 2019

  13. Ng, A.Y.: Shaping and Policy Search in Reinforcement Learning. Ph.D. thesis, University of California, Berkeley (2003)

    Google Scholar 

  14. Noer, D.: Parallelization of the Value-Iteration algorithm for Partially Observable Markov Decision Processes. Master’s thesis, Technical University of Denmark (2013)

    Google Scholar 

  15. Poupart, P.: Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes. Ph.D. thesis, University of Toronto (2005)

    Google Scholar 

  16. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1st edn. Wiley, New York (2005)

    MATH  Google Scholar 

  17. Ruiz, S., Hernandez, B.: A parallel solver for Markov decision process in crowd simulations. In: Proceedings of the Fourteenth Mexican International Conference on Artificial Intelligence, pp. 107–116 (2015)

    Google Scholar 

  18. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Pearson, London (2009)

    MATH  Google Scholar 

  19. Sanner, S.: Relational dynamic influence diagram language (RDDL): Language description (2010)

    Google Scholar 

  20. Sapio, A., Bhattacharyya, S., Wolf, M.: Efficient solving of Markov decision processes on GPUs using parallelized sparse matrices. In: Proceedings of the Conference on Design and Architectures for Signal and Image Processing (DASIP) (2018)

    Google Scholar 

  21. Sapio, A., Li, L., Wu, J., Wolf, M., Bhattacharyya, S.S.: Reconfigurable digital channelizer design using factored Markov decision processes. J. Sig. Process. Syst. 90(10), 1329–1343 (2018)

    Article  Google Scholar 

  22. Sapio, A., Wolf, M., Bhattacharyya, S.S.: Compact modeling and management of reconfiguration in digital channelizer implementation. In: Proceedings of the IEEE Global Conference on Signal and Information Processing, pp. 595–599 (2016)

    Google Scholar 

  23. Sim, H.S., Kim, K.E., Kim, J.H., Chang, D.S., Koo, M.W.: Symbolic heuristic search value iteration for factored POMDPs. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2008)

    Google Scholar 

  24. Smith, T.: Probabilistic Planning for Robotic Exploration. Ph.D. thesis, Carnegie Mellon University (2007)

    Google Scholar 

  25. Svalorzen, E.: AI-Toolbox. https://github.com/Svalorzen/AI-Toolbox. Accessed 05 Jan 2019

  26. Wei, Y., Wang, X., Guo, F., Hogan, G., Collier, M.: Energy saving local control policy for green reconfigurable routers. In: IEEE International Conference on Communications, pp. 221–225 (2015)

    Google Scholar 

  27. Younes, H.L.S., Littman, M.L.: PPDDL1.0: the language for the probabilistic part of IPC-4. Technical report

    Google Scholar 

Download references

Acknowledgments

This research was sponsored in part by the US National Science Foundation (CNS1514425 and CNS151304).

Author information

Authors and Affiliations

  1. University of Maryland, College Park, MD, USA

    Adrian E. Sapio, Rocky L. Tatiefo & Shuvra S. Bhattacharyya

  2. Georgia Institute of Technology, Georgia, USA

    Marilyn Wolf

Authors
  1. Adrian E. Sapio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rocky L. Tatiefo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuvra S. Bhattacharyya
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marilyn Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian E. Sapio .

Editor information

Editors and Affiliations

  1. Technical University of Crete and ICS - FORTH, Chania, Greece

    Dionisios N. Pnevmatikatos

  2. INSA Rennes, Rennes Cedex 7, France

    Maxime Pelcat

  3. Fraunhofer IESE, Kaiserslautern, Germany

    Matthias Jung

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sapio, A.E., Tatiefo, R.L., Bhattacharyya, S.S., Wolf, M. (2019). GEMBench: A Platform for Collaborative Development of GPU Accelerated Embedded Markov Decision Systems. In: Pnevmatikatos, D., Pelcat, M., Jung, M. (eds) Embedded Computer Systems: Architectures, Modeling, and Simulation. SAMOS 2019. Lecture Notes in Computer Science(), vol 11733. Springer, Cham. https://doi.org/10.1007/978-3-030-27562-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27562-4_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27561-7

  • Online ISBN: 978-3-030-27562-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation