Abstract
Lifelong Machine Learning (LML) has been receiving more and more attention in the past few years. It produces systems that are able to learn knowledge from consecutive tasks and refine the learned knowledge for a life time. In the optimization process of classical full-supervised LML systems, sufficient labeled data are required for extracting inter-task relationships before transferring. In order to leverage abundant unlabeled data and reduce the expenditure of labeling data, an progressive lifelong learning algorithm (PLLA) is proposed in this paper with unsupervised pre-training to learn shared representations that are more suitable as input to LML systems than the raw input data. Experiments show that the proposed PLLA is much more effective than many other LML methods when few labeled data is available.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Silver, D.L., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: AAAI Spring Symposium: Lifelong Machine Learning (2013)
Ammar, H.B., Eaton, E., Ruvolo, P., Taylor, M.: Online multi-task learning for policy gradient methods. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1206–1214 (2014)
Banko, M., Etzioni, O.: Strategies for lifelong knowledge extraction from the web. In: Proceedings of the 4th International Conference on Knowledge Capture, pp. 95–102 (2007)
Hong, R., Yang, Y., Wang, M., Hua, X.S.: Learning visual semantic relationships for efficient visual retrieval. IEEE Trans. Big Data 1, 152–161 (2017)
Chen, Z., Liu, B.: Topic modeling using topics from many domains, lifelong learning and big data. In: International Conference on International Conference on Machine Learning, pp. 703–711 (2014)
Hong, R., Hu, Z., Wang, R., Wang, M., Tao, D.: Multi-view object retrieval via multi-scale topic models. IEEE Trans. Image Process. 25, 5814–5827 (2016)
Hong, R., Zhang, L., Zhang, C., Zimmermann, R.: Flickr circles: aesthetic tendency discovery by multi-view regularized topic modeling. IEEE Trans. Multimedia 18, 1555–1567 (2016)
Ruvolo, P., Eaton, E.: Ella: an efficient lifelong learning algorithm. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13), June 2013
Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 521–528 (2011)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)
Silver, D.L., Mercer, R.E.: The task rehearsal method of life-long learning: overcoming impoverished data. In: Cohen, R., Spencer, B. (eds.) AI 2002. LNCS (LNAI), vol. 2338, pp. 90–101. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47922-8_8
Alsharif, O., Bachman, P., Pineau, J.: Lifelong learning of discriminative representations. CoRR (2014)
Alsharif, O., Bachman, P., Pineau, J.: Representation as a service. arXiv preprint arXiv:1404.4108 (2014)
Archambeau, C., Guo, S., Zoeter, O.: Sparse bayesian multi-task learning. In: Advances in Neural Information Processing Systems, pp. 1755–1763 (2011)
Evgeniou, A., Pontil, M.: Multi-task feature learning. Adv. Neural Inf. Process. Syst. 19, 41 (2007)
Bakker, B., Heskes, T.: Task clustering and gating for bayesian multitask learning. J. Mach. Learn. Res. 4, 83–99 (2003)
Mishra, M., Huan, J.: Learning task grouping using supervised task space partitioning in lifelong multitask learning. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 1091–1100. ACM (2015)
Kumar, A., Daume III, H.: Learning task grouping and overlap in multi-task learning. ArXiv e-prints (2012)
Ruvolo, P., Eaton, E.: Online multi-task learning via sparse dictionary optimization. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI-14) (2014)
Ruvolo, P., Eaton, E.: Active task selection for lifelong machine learning. In: Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI-13) (2013)
Wang, B., Pineau, J.: Online boosting algorithms for anytime transfer and multitask learning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Xue, Y., Liao, X., Carin, L., Krishnapuram, B.: Multi-task learning for classification with dirichlet process priors. J. Mach. Learn. Res. 8, 35–63 (2007)
Acknowledgements
This work is supported in part by the National Natural Science Founding of China (61171142, 61401163, U1636218), Science and Technology Planning Project of Guangdong Province of China (2014B010111003, 2014B010111006), the Fundamental Research Funds for the Central Universities (2017MS045), and Guangzhou Key Lab of Body Data Science (201605030011).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Su, G., Xu, X., Chen, C., Cai, B., Qing, C. (2018). Progressive Lifelong Learning by Sharing Representations for Few Labeled Data. In: Huet, B., Nie, L., Hong, R. (eds) Internet Multimedia Computing and Service. ICIMCS 2017. Communications in Computer and Information Science, vol 819. Springer, Singapore. https://doi.org/10.1007/978-981-10-8530-7_40
Download citation
DOI: https://doi.org/10.1007/978-981-10-8530-7_40
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-8529-1
Online ISBN: 978-981-10-8530-7
eBook Packages: Computer ScienceComputer Science (R0)