This paper proposes a simple image processing approach for detecting and tracking objects in the surrounding environment purporting to drive a portable exoskeleton for upper extremity rehabilitation. An experiment was carried out to test the feasibility of the proposed approach. The experiment was to mimic reaching an object placed on a ruler board in front of the user in the sagittal plane. Kinematical relations between the object and the user's hand were derived. The output of the image processing approach and the kinematical relations were used to generate a control signal used to drive a portable vision-assisted exoskeleton in a functional rehabilitation tasks. Results of the experimental show that the image processing approach and the kinematic equations are sufficient to drive an exoskeleton in a reaching task.