A novel user interface design is presented for semi-autonomous teleoperation system consisting of 3-DOF manipulator, 1-DOF linear stage, and flexible beam erected vertically. An LQR(Linear Quadratic Regulator) control designed for vibration suppression of the beam is applied to the slave-robot system autonomously, and a 3-DOF manipulator performs peg-in-hole tasks by the master in a remote place through inverse kinematics-based workspace tracking control. 3D vision transmitted to the master via HMD(Head Mounted Display) device and haptic feedback calculated from contact information are main components of our interface. From the perspective of human perception, optimization for 3D camera placement is performed to maximize human work efficiency. To verify the proposed user interface performance, human subject tests are performed for eight participants by comparing our interface with a 2D webcam interface and/or with a non-haptic feedback interface. Furthermore, in order to make it possible to run the robotic system in any environment, system on-board implementation by using three strain gauges, one IMU (Inertial Measurement Unit), and encoders of actuators is progressed in this paper.