In this work, we address the problem of online learning, where models must be continually updated from an incoming stream of data, while retaining past information. We develop an approach that is nonparametric, models uncertainty, and requires minimal hand-tuning. Our proposed algorithm, which we term online generalized product of experts (OGPoE), extends the powerful generalized product of experts (GPoE) framework to the online setting by leveraging methods for sparse, variational Gaussian process approximations, as well as nonparametric clustering. We devise a 1-D example learning problem to illustrate how our method works, and we verify that we achieve competitive results with other popular modeling approaches on a benchmark learning problem. Finally, we demonstrate how our algorithm can produce high accuracy predictions on a physical system, by learning the kinematics for a concentric tube robot, even when the robot is subject to changing, unknown loads.