Field robots used in unstructured and dynamic environments - and teleoperation have shifted into the focus of a variety of industrial branches in the past few years. The lack of space in the atomic industry, on oil platforms and in space applications demands additional adaptations to current robotic setups. In this paper a MMSS (Multi-Master-Single-Slave) haptic teleoperation system is proposed through which one operator using two master arms can manipulate objects in a cooperative way via one slave robot and a virtual gripping point. To ease the execution of a peg-in-hole task of big objects, a task allocation in the cartesian frame of the virtual gripping point is introduced additionally. The stability of this multilateral system with time delay is guaranteed by the Time Domain Passivity Approach. Therefore the system is divided into several modular subsystems which renders the system easily adaptable to other scenarios.