Particle swarm optimization (PSO) is a nature-inspired technique for solving continuous optimization problems. For a fixed optimization problem, the quality of the found solution depends significantly on the choice of the algorithmic PSO parameters such as the inertia weight and the acceleration coefficients. It is a challenging task to choose appropriate values for these parameters by hand or mathematically. In this paper, a novel self-optimizing particle swarm optimizer with multiple adaptation layers is introduced. In the new algorithm, adaptation takes place on both particle and subswarm level. The new idea of using virtual parameter swarms which hold modifiable parameter configurations each is introduced. The algorithmic PSO parameters can be mutated by using, for instance, well-known techniques from the field of evolutionary algorithms, in order to allow fine-granular parameter adaptation to the problem at hand. The new algorithm is experimentally evaluated, and compared to a standard PSO and the Tribes algorithm. The experimental study shows that our new algorithm is highly competitive to previously suggested approaches.