Handing over objects is an essential task in human-robot collaborative scenarios. Previous studies have predominantly employed rigid grippers to perform the handover, focusing on generating grasps that avoid physical contact with people. In this paper, we present a vision-based open-palm handover solution where a soft robotic hand exploits contact with the human hand for improved grasp success and robustness. The human-robot physical interaction allows the robotic hand to slide over the human palm and firmly cage the object. The identification of the human hand plane and object pose is achieved through a versatile perception pipeline that exploits a single RGB-D camera. Through experimental trials, we show that the system achieves successful grasps over multiple objects with different geometries and textures. A comparative analysis assesses the robustness of the proposed soft handover method against a baseline approach. A study with 30 participants evaluates users' perception of human-robot interaction during the handover, highlighting the effectiveness and preference for the proposed pipeline.