Abstract
We propose a sparse Gaussian process model to approximate full Gaussian process with derivatives when a large number of function observations t and derivative observations \(t'\) exist. By introducing a small number of inducing point m, the complexity of posterior computation can be reduced from \(\mathcal {O}((t+t')^{3})\) to \(\mathcal {O}((t+t')m^{2})\). We also find the usefulness of our approach in Bayesian optimisation. Experiments demonstrate the superiority of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Riihimäki, J., Vehtari, A.: Gaussian processes with monotonicity information. In: Proceedings of the Thirteenth International Conference on AIStat (2010)
Siivola, E., Vehtari, A., Vanhatalo, J., González, J.: Bayesian optimization with virtual derivative sign observations. arXiv preprint arXiv:1704.00963 (2017)
Solak, E., Murray-Smith, R., Leithead, W.E., Leith, D.J., Rasmussen, C.E.: Derivative observations in Gaussian process models of dynamic systems. In: NIPS (2003)
Jauch, M., Peña, V.: Bayesian optimization with shape constraints. arXiv preprint arXiv:1612.08915 (2016)
Li, C., Rana, S., Gupta, S., Nguyen, V., Venkatesh, S.: Bayesian optimization with monotonicity information (2017)
Yang, A., Li, C., Rana, S., Gupta, S., Venkatesh, S.: Efficient Bayesian optimisation using derivative meta-model. In: Geng, X., Kang, B.-H. (eds.) PRICAI 2018. LNCS (LNAI), vol. 11013, pp. 256–264. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97310-4_29
Snelson, E., Ghahramani, Z.: Sparse gaussian processes using pseudo-inputs. In: Advances in Neural Information Processing Systems, pp. 1257–1264 (2006)
Snelson, E., Ghahramani, Z.: Variable noise and dimensionality reduction for sparse Gaussian processes. arXiv preprint arXiv:1206.6873 (2012)
Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6(Dec), 1939–1959 (2005)
Li, C., Gupta, S., Rana, S., Nguyen, V., Venkatesh, S., Shilton, A.: High dimensional Bayesian optimization using dropout. arXiv preprint arXiv:1802.05400 (2018)
Rana, S., Li, C., Gupta, S., Nguyen, V., Venkatesh, S.: High dimensional Bayesian optimization with elastic gaussian process. In: International Conference on Machine Learning, pp. 2883–2891 (2017)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)
Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning. MIT press, Cambridge (2006)
Brochu, E., Cora, V.M., De Freitas, N.: A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 (2010)
Mockus, J.: Application of Bayesian approach to numerical methods of global and stochastic optimization. J. Glob. Optim. 4(4), 347–365 (1994)
Kushner, H.J.: A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Basic Eng. 86(1), 97–106 (1964)
Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. arXiv preprint arXiv:0912.3995 (2009)
Finkel, D.E.: Direct optimization algorithm user guide. CRSC 2, 1–14 (2003)
Acknowledgment
This research was partially funded by the Australian Government through the Australian Research Council (ARC). Prof Venkatesh is the recipient of an ARC Australian Laureate Fellowship (FL170100006).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, A., Li, C., Rana, S., Gupta, S., Venkatesh, S. (2018). Sparse Approximation for Gaussian Process with Derivative Observations. In: Mitrovic, T., Xue, B., Li, X. (eds) AI 2018: Advances in Artificial Intelligence. AI 2018. Lecture Notes in Computer Science(), vol 11320. Springer, Cham. https://doi.org/10.1007/978-3-030-03991-2_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-03991-2_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03990-5
Online ISBN: 978-3-030-03991-2
eBook Packages: Computer ScienceComputer Science (R0)