To achieve more dexterous surgical operations, development of robots with more degrees of freedom (DOFs) is becoming one of the main trends for surgical robot research. However, manipulation complexity is therefore increased and it is difficult for surgeons to control all the DOFs with two hands alone. In this letter, we proposed a wearable hands-free human-robot interface to simultaneously control 4 DOFs of the flexible endoscope for the adjustment of the field of view. Motion signals of the trunk and one foot were collected with two inertial measurement units (IMUs), and then mapped to control signals of the endoscope. Performance of the proposed interface was firstly evaluated through virtual object control. Then a robotic platform was developed to further verify the feasibility of the hands-free interface for real endoscope control. Results showed that subjects were able to control the endoscope in an intuitive way through volitional motions of the trunk and foot in surgery-mimicked experiment tasks, and successfully and solely completed operations that required at least two subjects in traditional operation manners. The results indicate the promise of adopting the proposed hands-free interface in real surgical scenarios. We believe this study could also provide new ideas for more efficient control of other multi-DOF surgical robots.