This article introduces a probabilistic machine learning framework for the uncertainty quantification (UQ) of electronic circuits based on the Gaussian process regression (GPR). As opposed to classical surrogate modeling techniques, GPR inherently provides information on the model uncertainty. The main contribution of this work is twofold. First, it describes how, in a UQ scenario, the model uncertainty can be combined with the uncertainty of the input design parameters to provide confidence bounds for the statistical estimates of the system outputs, such as moments and probability distributions. These confidence bounds allow assessing the accuracy of the predicted statistics. Second, in order to deal with dynamic multioutput systems, principal component analysis (PCA) is effectively employed to compress the time-dependent output variables into a smaller set of components, for which the training of individual GPR models becomes feasible. The uncertainty on the principal components is then propagated back to the original output variables. Several application examples, ranging from a trivial RLC circuit to real-life designs, are used to illustrate and validate the advocated approach.