Abstract
Games on the Atari 2600 platform have served as a benchmark for reinforcement learning algorithms in recent years, and while deep reinforcement learning approaches make progress on most games, there are still some games that the majority of these algorithms struggle with. These are called hard exploration games. We introduce two new developments for the Random Network Distillation (RND) architecture. We apply self-attention and the mechanism of ego motion on the RND architecture and we evaluate them on three hard exploration tasks from the Atari platform. We find that the proposed ego network model improve the baseline of the RND architecture on these tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aytar, Y., Pfaff, T., Budden, D., Paine, T., Wang, Z., de Freitas, N.: Playing hard exploration games by watching YouTube. In: NeurIPS (2018)
Badia, A.P., et al.: Agent57: outperforming the Atari human benchmark. ArXiv abs/2003.13350 (2020)
Badia, A.P., et al.: Never give up: learning directed exploration strategies. ArXiv abs/2002.06038 (2020)
Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environment: an evaluation platform for general agents (extended abstract). J. Artif. Intell. Res. 47, 253–279 (2013)
Bellemare, M.G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., Munos, R.: Unifying count-based exploration and intrinsic motivation. ArXiv abs/1606.01868 (2016)
Bello, I., Zoph, B., Vaswani, A., Shlens, J., Le, Q.V.: Attention Augmented Convolutional Networks. arXiv e-prints arXiv:1904.09925, April 2019
Burda, Y., Edwards, H., Storkey, A.J., Klimov, O.: Exploration by random network distillation. CoRR abs/1810.12894 (2018). http://arxiv.org/abs/1810.12894
Burda, Y., Edwards, H.A., Pathak, D., Storkey, A.J., Darrell, T., Efros, A.A.: Large-scale study of curiosity-driven learning. ArXiv abs/1808.04355 (2019)
Choi, J., Lee, B., Zhang, B.: Multi-focus attention network for efficient deep reinforcement learning. ArXiv abs/1712.04603 (2017)
Choi, J., et al.: Contingency-aware exploration in reinforcement learning. ArXiv abs/1811.01483 (2019)
Haber, N., Mrowca, D., Fei-Fei, L., Yamins, D.L.K.: Emergence of structured behaviors from curiosity-based intrinsic motivation. CoRR abs/1802.07461 (2018). http://arxiv.org/abs/1802.07461
Horgan, D., et al.: Distributed prioritized experience replay. ArXiv abs/1803.00933 (2018)
Kapturowski, S., Ostrovski, G., Dabney, W., Quan, J., Munos, R.: Recurrent experience replay in distributed reinforcement learning. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=r1lyTjAqYX
Martin, J., Sasikumar, S.N., Everitt, T., Hutter, M.: Count-based exploration in feature space for reinforcement learning. In: IJCAI (2017)
Mott, A., Zoran, D., Chrzanowski, M., Wierstra, D., Rezende, D.J.: Towards interpretable reinforcement learning using attention augmented agents. In: NeurIPS (2019)
Ostrovski, G., Bellemare, M.G., Oord, A., Munos, R.: Count-based exploration with neural density models. ArXiv abs/1703.01310 (2017)
Ostrovski, G., Bellemare, M.G., van den Oord, A., Munos, R.: Count-based exploration with neural density models. ArXiv abs/1703.01310 (2017)
Schrittwieser, J., et al.: Mastering Atari, go, chess and shogi by planning with a learned model. ArXiv abs/1911.08265 (2019)
Sorokin, I., Seleznev, A., Pavlov, M., Fedorov, A., Ignateva, A.: Deep attention recurrent Q-network. ArXiv abs/1512.01693 (2015)
Tang, H., et al.: Exploration: a study of count-based exploration for deep reinforcement learning. In: NIPS (2017)
Tang, Y., Nguyen, D., Ha, D.: Neuroevolution of self-interpretable agents. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Marin, A., Rebedea, T., Hosu, I. (2021). Ego Networks. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13108. Springer, Cham. https://doi.org/10.1007/978-3-030-92185-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-92185-9_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92184-2
Online ISBN: 978-3-030-92185-9
eBook Packages: Computer ScienceComputer Science (R0)