Different robotic manipulation tasks require different execution and planning strategies. Nevertheless, the versatility of tasks in assembly and disassembly demands flexible control strategies. Fundamental to achieving such adaptive control methods is understanding and generalizing the interactions between tools, the manipulated object, and the environment required to perform a manipulation. This paper addresses the problem of generating adaptive manipulation by introducing the force-velocity task phase plot that represents the inherent nature of tactile manipulation skills. This representation enables us to identify the primary phases of the interaction in the force-velocity domain. Using unified force-impedance control, we establish a tactile manipulation strategy to robustly conduct versatile manipulation tasks even in case of disturbances or imprecise task information. The proposed control scheme features a dynamic process for impedance shaping based on the external force applied to the robot and the skill motion error for collision response, as well as a force-shaping function that enables both a smooth transition from free motion to contact and force regulation. We implement and compare the control strategy to previously proposed strategies using peg-in-hole reference experiments that include force disturbance and positioning inaccuracies and show the respective task phase plots. As a result, we observe high controller robustness and conclude that using the task phase plot as the inherent representation of tactile manipulation via unified force-impedance control enables successful adaptive controller design and creates a quantifiable basis for robotic skill solution comparison.