This study develops a fault estimation and accommodation scheme for the image-based visual servoing system to eliminate the effects of the faults due to the image feature extraction task, which is named as bias virtual sensor fault. First, a bias virtual sensor fault in visual servoing is declared. Then, fault diagnosis, which includes fault detection, isolation, and estimation, is designed based on the means of particle filter. Finally, a fault accommodation law is developed based on the information obtained from the fault estimation to compensate for the effects of the fault in the system. The proposed fault estimation and accommodation is verified through simulation and experimental studies, and the results show that the system can estimate and eliminate the unknown fault effects effectively.