Teleoperation systems are inefficient due to operator errors and system delays. Moreover, complexity is even higher for teleoperation tasks that involve repetitions. In this paper a technique that consists of automatic learning of operator commands and scene objects is proposed. Exploiting learned action patterns and scene objects, system is able to take the appropriate actions for it's current state. A number of experiments prepared to assess the proposed system. Experimental results show that the system can solve some known problems of teleoperation and increase the throughput of the operators.