Abstract
Deep Neural Networks (DNNs) are often criticized because they lack the ability to learn more than one task at a time: Multitask Learning is an emerging research area whose aim is to overcome this issue. In this work, we introduce the Pareto Multitask Learning framework as a tool that can show how effectively a DNN is learning a shared representation common to a set of tasks. We also experimentally show that it is possible to extend the optimization process so that a single DNN simultaneously learns how to master two or more Atari games: using a single weight parameter vector, our network is able to obtain sub-optimal results for up to four games.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Auger, A., Bader, J., Brockhoff, D., Zitzler, E.: Hypervolume-based multiobjective optimization: theoretical foundations and practical implications. Theoret. Comput. Sci. 425, 75–103 (2012)
Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environment: an evaluation platform for general agents. J. Artif. Intell. Res. 47, 253–259 (2013)
Brockman, G., et al.: Openai gym (2016)
Caruana R.: Multitask learning. In: Thrun S., Pratt L. (eds) Learning to Learn, pp. 95–133. Springer, Boston (1998). https://doi.org/10.1007/978-1-4615-5529-2_5
Conti, E., Madhavan, V., Petroski Such, F., Lehman, J., Stanley, K.O., Clune, J.: Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. In: NeurIPS 2018, Montreal, Canada (2018)
Fonseca, C.M., Paquete, L., López-Ibáñez, M.: An improved dimension-sweep algorithm for the hypervolume indicator. In: 2006 IEEE International Conference on Evolutionary Computation, pp. 1157–1163 (2006)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016) http://www.deeplearningbook.org
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Statist. 22(1), 79–86 (1951). https://doi.org/10.1214/aoms/1177729694
Kumar, M.P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 23, pp. 1189–1197. Curran Associates, Inc. (2010)
Maurer, A., Pontil, M., Romera-Paredes, B.: Sparse coding for multitask and transfer learning. In: Dasgupta, S., McAllester, D. (eds.) Proceedings of the 30th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 28, pp. 343–351. PMLR, Atlanta, Georgia, USA, 17–19 June 2013. http://proceedings.mlr.press/v28/maurer13.html
Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning (2016)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015). https://doi.org/10.1038/nature14236
Murugesan, K., Carbonell, J.: Self-paced multitask learning with shared knowledge. IJCAI-17 (2017)
Romera-Paredes, B., Aung, H., Bianchi-Berthouze, N., Pontil, M.: Multilinear multitask learning. In: Dasgupta, S., McAllester, D. (eds.) Proceedings of the 30th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 28, pp. 1444–1452. PMLR, Atlanta, Georgia, USA, 17–19 June 2013. http://proceedings.mlr.press/v28/romera-paredes13.html
Ruder, S.: An overview of multi-task learning in deep neural networks. CoRR (2017)
Salimans, T., Ho, J., Chen, X., Sidor, S., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. arXiv e-prints arXiv:1703.03864, March 2017
Schmidhuber, J.: Ultimate cognition à la gödel. Cognitive Comput. 1(2), 177–193 (2009). https://doi.org/10.1007/s12559-009-9014-y
Silver, D., et al.: A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science 362, 1140–1144 (2018)
Stanley, K., Clune, J., Lehman, J., Miikkulainen, R.: Designing neural networks through neuroevolution. Nat. Mach. Intell. (2019). https://doi.org/10.1038/s42256-018-0006-z
Stracquadanio, G., Nicosia, G.: Computational energy-based redesign of robust proteins. Comput. Chem. Eng. (2010). https://doi.org/10.1016/j.compchemeng.2010.04.005
Zhang, Y., Yang, Q.: An overview of multi-task learning. Nat. Sci. Rev. 5(1), 30–43 (2018). https://doi.org/10.1093/nsr/nwx105
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dyankov, D., Riccio, S.D., Di Fatta, G., Nicosia, G. (2019). Multi-task Learning by Pareto Optimality. In: Nicosia, G., Pardalos, P., Umeton, R., Giuffrida, G., Sciacca, V. (eds) Machine Learning, Optimization, and Data Science. LOD 2019. Lecture Notes in Computer Science(), vol 11943. Springer, Cham. https://doi.org/10.1007/978-3-030-37599-7_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-37599-7_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37598-0
Online ISBN: 978-3-030-37599-7
eBook Packages: Computer ScienceComputer Science (R0)