Bayesian optimization with Gaussian Process (GP) models has been proposed for analog synthesis since it is efficient for the optimizations of expensive black-box functions. However, the computational cost for training and prediction of Gaussian process models are O(N³) and O(N²), respectively, where N is the number of data points. The overhead of the Gaussian process modeling would not be negligible as N is relatively large. Recently, a Bayesian optimization approach using neural network has been proposed to address this problem. It reduces the computational cost of training and prediction of Gaussian process models to O(N) and O(1), respectively. However, reducing the infinite-dimensional kernel to finite-dimensional kernel using neural network mapping would weaken the characterization ability of Gaussian process. In this paper, we propose a novel Bayesian optimization approach using Sparse Pseudo-input Gaussian Process (SPGP). The idea is to use M <; N so-called inducing points to build a sparse Gaussian process model to approximate the conventional exact Gaussian process model. Without the need to sacrifice the modeling ability of the surrogate model, it also reduces the computational cost of both training and prediction to O(N) and O(1), respectively. Several experiments were provided to demonstrate the efficiency of the proposed approach.