Douglas Cale Crowder
ABSTRACT
Innovation in the field of neuromorphic computing is characterized by long periods of slow, steady growth that are punctuated by periods of rapid discovery. In order to accelerate discovery in the field of neuromorphic computing and disrupt the process of slow and steady growth, we are proposing a reinforcement learning architecture that is able to automatically construct simple circuits. Rather than providing the reinforcement learning agent with specifications for existing devices, we ask the reinforcement learning agent to provide specifications for novel devices that, if fabricated, can solve a specified problem. We show that, by slightly changing the problem statement, we can cause the reinforcement learning agent to produce different device specifications. Ultimately, we expect that by generating many possible solutions, the reinforcement learning agent will accelerate innovation by stimulating insight into potential solutions for problems.
- Muhammad Abrar, Ushna Ajmal, Ziyad M Almohaimeed, Xiang Gui, Rizwan Akram, and Roha Masroor. 2021. Energy efficient UAV-enabled mobile edge computing for IoT devices: a review. IEEE Access (2021).Google Scholar
Cross Ref
- James Aimone, Prasanna Date, Gabriel Fonseca-Guerra, Kathleen Hamilton, Kyle Henke, Bill Kay, Garrett Kenyon, Shruti Kulkarni, Susan Mniszewski, and Maryam Parsa. 2022. A review of non-cognitive applications for neuromorphic computing. Neuromorphic Computing and Engineering (2022).Google Scholar
- Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. 2017. Hindsight experience replay. Advances in neural information processing systems 30 (2017).Google ScholarDigital Library
- Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando G. S. L. Brandao, David A. Buell, Brian Burkett, Yu Chen, Zijun Chen, Ben Chiaro, Roberto Collins, William Courtney, Andrew Dunsworth, Edward Farhi, Brooks Foxen, Austin Fowler, Craig Gidney, Marissa Giustina, Rob Graff, Keith Guerin, Steve Habegger, Matthew P. Harrigan, Michael J. Hartmann, Alan Ho, Markus Hoffmann, Trent Huang, Travis S. Humble, Sergei V. Isakov, Evan Jeffrey, Zhang Jiang, Dvir Kafri, Kostyantyn Kechedzhi, Julian Kelly, Paul V. Klimov, Sergey Knysh, Alexander Korotkov, Fedor Kostritsa, David Landhuis, Mike Lindmark, Erik Lucero, Dmitry Lyakh, Salvatore Mandrà, Jarrod R. McClean, Matthew McEwen, Anthony Megrant, Xiao Mi, Kristel Michielsen, Masoud Mohseni, Josh Mutus, Ofer Naaman, Matthew Neeley, Charles Neill, Murphy Yuezhen Niu, Eric Ostby, Andre Petukhov, John C. Platt, Chris Quintana, Eleanor G. Rieffel, Pedram Roushan, Nicholas C. Rubin, Daniel Sank, Kevin J. Satzinger, Vadim Smelyanskiy, Kevin J. Sung, Matthew D. Trevithick, Amit Vainsencher, Benjamin Villalonga, Theodore White, Z. Jamie Yao, Ping Yeh, Adam Zalcman, Hartmut Neven, and John M. Martinis. 2019. Quantum supremacy using a programmable superconducting processor. Nature 574, 7779 (2019), 505--510. Google ScholarCross Ref
- Geoffrey W Burr, Robert M Shelby, Abu Sebastian, Sangbum Kim, Seyoung Kim, Severin Sidler, Kumar Virwani, Masatoshi Ishii, Pritish Narayanan, Alessandro Fumarola, et al. 2017. Neuromorphic computing using non-volatile memory. Advances in Physics: X 2, 1 (2017), 89--124.Google Scholar
- Douglas C Crowder, J Darby Smith, and Suma G Cardwell. 2023. AI-Enhanced Codesign of Neuromorphic Circuit. In 2023 IEEE 65th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE.Google Scholar
- Steve Furber. 2016. Large-scale neuromorphic computing systems. Journal of neural engineering 13, 5 (2016), 051001.Google ScholarCross Ref
- Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al. 2020. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence 44, 1 (2020), 154--180.Google Scholar
- Kurt Hornik, Maxwell Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. Neural networks 2, 5 (1989), 359--366.Google ScholarDigital Library
- Andrew Jolley, Saeed Afshar, Greg Cohen, Richard Lazarus Pahlavani, and Andrew Lambert. 2023. Neuromorphic Sensor Event-Rate Monitoring for Satellite Characterization. Journal of Spacecraft and Rockets (2023), 1--12.Google Scholar
- Norman Jouppi, Cliff Young, Nishant Patil, and David Patterson. 2018. Motivation for and Evaluation of the First Tensor Processing Unit. IEEE Micro 38, 3 (2018), 10--19. Google ScholarCross Ref
- Olga Krestinskaya, Alex Pappachen James, and Leon Ong Chua. 2019. Neuromemristive circuits for edge computing: A review. IEEE transactions on neural networks and learning systems 31, 1 (2019), 4--23.Google Scholar
- Suhas Kumar, R Stanley Williams, and Ziwen Wang. 2020. Third-order nanocircuit elements for neuromorphic engineering. Nature 585, 7826 (2020), 518--523.Google Scholar
- Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. 2021. A graph placement methodology for fast chip design. Nature 594, 7862 (2021), 207--212.Google Scholar
- Andrew Y Ng, Daishi Harada, and Stuart Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In Icml, Vol. 99. 278--287.Google ScholarDigital Library
- Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. 2021. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research 22, 268 (2021), 1--8.Google Scholar
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.Google ScholarDigital Library
- Chetan Singh Thakur, Jamal Lottier Molin, Gert Cauwenberghs, Giacomo Indiveri, Kundan Kumar, Ning Qiao, Johannes Schemmel, Runchun Wang, Elisabetta Chicca, Jennifer Olson Hasler, et al. 2018. Large-scale neuromorphic spiking array processors: A quest to mimic the brain. Frontiers in neuroscience 12 (2018), 891.Google Scholar
- Craig Vineyard, Suma Cardwell, Frances Chance, Srideep Musuvathy, Fred Rothganger, William Severa, John Smith, Corinne Teeter, Felix Wang, and James Aimone. 2022. Neural Mini-Apps as a Tool for Neuromorphic Computing Insight. In Neuro-Inspired Computational Elements Conference. 40--49.Google ScholarDigital Library
- Craig Vineyard, William Severa, Matthew Kagie, Andrew Scholand, and Park Hays. 2019. A Resurgence in Neuromorphic Architectures Enabling Remote Sensing Computation. In 2019 IEEE Space Computing Conference (SCC). IEEE, 33--40.Google ScholarCross Ref
- M Mitchell Waldrop. 2016. The chips are down for Moore's law. Nature News 530, 7589 (2016), 144.Google ScholarCross Ref
- Angel Yanguas-Gil, Jaehoon Koo, Sandeep Madireddy, Prasanna Balaprakash, Jeffrey W Elam, and Anil U Mane. 2021. Neuromorphic architectures for edge computing under extreme environments. In 2021 IEEE Space Computing Conference (SCC). IEEE, 39--45.Google ScholarCross Ref
Index Terms
- Deep Reinforcement Learning Methods for Discovering Novel Neuromorphic Devices
Recommendations
Discrete-to-deep reinforcement learning methods
AbstractNeural networks are effective function approximators, but hard to train in the reinforcement learning (RL) context mainly because samples are correlated. In complex problems, a neural RL approach is often able to learn a better solution than ...
Deep Reinforcement Learning: From Q-Learning to Deep Q-Learning
Neural Information ProcessingAbstractAs the two hottest branches of machine learning, deep learning and reinforcement learning both play a vital role in the field of artificial intelligence. Combining deep learning with reinforcement learning, deep reinforcement learning is a method ...
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent SystemsRecent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Comments