In this paper, we are presenting a new dynamically evolving clustering approach which extends conventional evolving Vector Quantization (eVQ), successfully applied before as fast learning engine for evolving cluster models, classifiers and evolving fuzzy systems in various real-world applications. The first extension concerns the ability to extract ellipsoidal prototype-based clusters in arbitrary position, thus increasing its flexibility to model any orentiation/rotation of local data clouds. The second extension includes a single-pass merging strategy in order to resolve unnecessary overlaps or to dynamically compensate inappropriately chosen learning parameters (which may lead to over-clustering effects). The new approach, termed as eVQ-AM (eVQ for Arbitrary ellipsoids with Merging functionality), is compared with conventional eVQ, other incremental and batch learning clustering methods based on two-dimensional as well as high-dimensional streaming clustering showing an evolving behavior in terms of adding/joining clusters as well as feature range expansions. The comparison includes a sensitivity analysis on the learning parameters and observations of finally achieved cluster partition qualities.