Abstract
Active learning is an effective method to reduce the learning time, space and economic costs in the whole training procedure. It aims to select more informative points from the unlabeled data pool, label them and add them into the training set, which helps to improve the performance of learning models. Learning models and active learning strategies are two essential elements in the framework of active learning. Probabilistic models such as Gaussian processes are often used as learning models for active learning, which have achieved promising results attributed to their predictive uncertainty. In order to well model complex data and characterize uncertainty, we employ deep Gaussian processes (DGPs) as learning models, based on which active learning strategies are made. Specifically, we design appropriate active learning strategies based on DGPs for solving binary and multi-class classification tasks, respectively. The experiments on educational and non-educational text classification and handwritten digit recognition demonstrate the effectiveness of the proposed active learning methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
\(\mathbf {z}_l \) will be omitted in our paper to simplify the notation.
- 2.
The \(q^{\setminus n}(\mathbf {u})\) is the variational cavity distribution of \(\mathbf {u}\) and \(q^{\setminus n}(\mathbf {u}) = q(\mathbf u)/\tilde{t}_n(\mathbf u)\).
- 3.
The \(q^{\setminus 1}(\mathbf {u})\) is the variational cavity distribution of \(\mathbf {u}\) and \(q^{\setminus 1}(\mathbf {u}) = q(\mathbf u)/g(\mathbf u).\).
- 4.
Word2vec is an efficient tool for Google to represent the words as real value vectors. The python program can be achieved using the gensim toolkit.
References
Huang, S., Jin, R., Zhou, Z.: Active learning by querying informative and representative examples. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 1936–1949 (2014)
Zhou, J., Sun, S.: Gaussian process versus margin sampling active learning. Neurocomputing 167(1), 122–131 (2015)
Gavves, E., Mensink, T., Tommasi, T., Snoek, C.G.M., Tuytelaars, T.: Active transfer learning with zero-shot priors: reusing past datasets for future tasks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2731–2739. IEEE, New York (2015)
Seeger, M.: Gaussian processes for machine learning. Int. J. Neural Syst. 14(2), 69–106 (2004)
Ma, Y., Sutherland, D., Garnett, R., Schneider, J.: Active pointillistic pattern search. In: Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, pp. 672–680. JMLR, Cambridge (2015)
Liu, Q., Sun, S.: Sparse multimodal Gaussian processes. In: Sun, Y., Lu, H., Zhang, L., Yang, J., Huang, H. (eds.) IScIDE 2017. LNCS, vol. 10559, pp. 28–40. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67777-4_3
Luo, C., Sun, S.: Variational mixtures of Gaussian processes for classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 4603–4609. Morgan Kaufmann, San Francisco (2017)
Mosinskadomanska, A., Sznitman, R., Glowack, P., Fua, P.: Active learning for delineation of curvilinear structures. In: Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, pp. 5231–5239. IEEE, New York (2015)
Lu, J., Zhao, P., Steven, S.C.H.: Online passive aggressive active learning and its applications. In: Proceedings of the Asian Conference on Machine Learning, pp. 266–282. JMLR, Cambridge (2014)
Yang, Y., Ma, Z., Nie, F., Chang, X., Hauptmann, A.G.: Multi-class active learning by uncertainty sampling with diversity maximization. Int. J. Comput. Vis. 113(2), 113–127 (2015)
Lewenberg, Y., Bachrach, Y., Paquet, U., Rosenschein, J.: Knowing what to ask: A Bayesian active learning approach to the surveying problem. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 1396–1402. AAAI, San Francisco (2017)
Rasmussen, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Nickisch, H., Rasmussen, C.: Approximations for binary Gaussian process classification. J. Mach. Learn. Res. 9(10), 2035–2078 (2008)
Kim, H., Ghahramani, Z.: Bayesian Gaussian process classification with the EM-EP algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 1948–1959 (2006)
Zhao, J., Sun, S.: Variational dependent multi-output Gaussian process dynamical systems. J. Mach. Learn. Res. 17(1), 1–36 (2016)
Lawrence, N., Moore, A.: Hierarchical Gaussian process latent variable models. In: Proceedings of the 24th International Conference on Machine Learning, pp. 481–488. ACM, New York (2007)
Damianou, A., Lawrence, N.: Deep Gaussian processes. In: Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, pp. 207–215. JMLR, Cambridge (2013)
Dai, Z., Damianou, A., González, J., Lawrence, N.: Variational auto-encoded deep Gaussian processes. Comput. Sci. 14(9), 3942–3951 (2015)
Bui, T., Hernández-Lobato, D., Hernandez-Lobato, J., Li, Y., Turner, R.: Deep Gaussian processes for regression using approximate expectation propagation. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 1472–1481. ACM, New York (2016)
Li, Y., Hernández-Lobato, J., Turner, R.: Stochastic expectation propagation. Adv. Neural Inf. Process. Syst. 28(1), 2323–2331 (2015)
Acknowledgments
The first two authors Jingjing Fei and Jing Zhao are joint first authors. The corresponding author is Shiliang Sun. This work is sponsored by NSFC Project 61673179, Shanghai Sailing Program, and Shanghai Knowledge Service Platform Project (No. ZF1213).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Fei, J., Zhao, J., Sun, S., Liu, Y. (2018). Active Learning Methods with Deep Gaussian Processes. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11303. Springer, Cham. https://doi.org/10.1007/978-3-030-04182-3_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-04182-3_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04181-6
Online ISBN: 978-3-030-04182-3
eBook Packages: Computer ScienceComputer Science (R0)