We developed a new hand model for increasing the robustness of finger-based manipulations of virtual objects. Each phalanx of our hand model consists of a number of deformable soft bodies, which dynamically adapt to the shape of grasped objects based on the applied forces. Stronger forces directly result in larger contact areas, which increase the friction between hand and object as would occur in reality. For a robust collision-based soft body simulation, we extended the lattice-shape matching algorithm to work with adaptive stiffness values, which are dynamically derived from force and velocity thresholds. Our implementation demonstrates that this approach allows very precise and robust grasping, manipulation and releasing of virtual objects and performs in real-time for a variety of complex scenarios. Additionally, laborious tuning of object and friction parameters is not necessary for the wide range of objects that we typically grasp with our hands.