Prosthesis is a device that replaces a missing body part. These devices are used to reinstate the lost functionality due to an amputation. Above elbow amputees are equipped with trans-humeral prostheses. The existing prostheses use biological signals such as electromyography (EMG) in combine with external sensors for the controlling purposes. Vision is the latest addition to external sensor domain to eliminate the drawbacks of existing hybrid prostheses. This paper proposes a vision based task planning method in combination with available EMG signals to plan reach-to-grasp tasks of a trans-humeral prosthesis. The visual servoing system consists of an Image Based Visual Servoing (IBVS) system in conjunction with an Artificial Neural Network (ANN) to center the object in the image frame. Moreover, a gradient descent algorithm is proposed for the elbow motion to reach the palm towards the object of interest. EMG based system is capable of controlling the prosthetic elbow motion with available muscles' signals. EMG based system is fused with the IBVS system to perform elbow motions when an object is visible to the camera. Experiments are carried out using a virtual prosthesis to validate the proposed task planning method.