One of the greatest challenges of life-long learning architectures is how to efficiently and reliably cope with the stability-plasticity dilemma. We propose an extension of a flexible system combining a static offline classifier and an incremental online classifier that is well suited for life-long learning scenarios. The pre-trained offline classifier preserves ground knowledge that should be respected during training, while the online classifier enables learning of new or specific information encountered during use. The combination is realised by a dynamic classifier selection strategy based on confidences of both ingredients. We report exemplary results of this architecture for the case of learning vector quantization (LVQ) for several data sets, thereby including an extensive comparison to alternative state of the art algorithms for incremental learning such as incremental generalised LVQ and the support vector machine.