The probabilistic classification vector machine is a very effective and generic probabilistic and sparse classifier. A recently published incremental version improved the runtime complexity to quadratic costs. We derive the Nyström approximation for asymmetric matrices to obtain linear runtime and memory complexity for the incremental probabilistic classification vector machine while keeping similar prediction performance.