Selecting an appropriate object transfer point can effectively improve human comfort in the process of human-robot object transfer. In this article, a human-robot handover system based on human behavior patterns is proposed. First, the arm joint torque model and the medium joint angle model were combined to establish the human comfort model, and a binary cost function was constructed to predict the object transfer points for human-robot object transfer. Skeleton and RGB-D information were fused to construct a discriminant model of intention transfer for human-robot object transfer. Then the accuracy of intent recognition was verified through transfer intention recognition experiment. A human-to-human handover experiment was designed to obtain the actual object transfer points based on the OpenPose skeleton recognition, and the Bonferroni method was used to verify the difference between the predicted object transfer points and the actual transfer points, which proved that the predicted object transfer point was consistent with the actual transfer point. Finally, the experiments of robot-to-human handover and human-to-robot handover were carried out. The results of multipoint comparison surveys showed that the model can predict object transfer points that conform to human handover habits and bring more natural and smooth transfer experiences to human interactors.