Multi-task Semi-supervised Learning Framework for Surgical Instrument Pose Estimation

As an important part of medical technology, the process of instrumental surgery has various risks, which can be reduced by using computer assisted intervention system during surgery on operation and decision making, while surgical instrument pose estimation is an important part in such system. The constraint of instrument pose estimation has always been sparse data annotation in endoscopic scenario. We propose a semi-supervised and multi-task learning based pose estimation framework to enhance the model effect using unannotated data. Due to the severe class imbalance in instrument pose estimation, simple semi-supervised learning results in model degradation, where network tends to classify every pixel as background. In this paper, we combine multi-task and semi-supervised learning, using instrument segmentation as a spatial constraint and instrument number detection as a global constraint, thus providing additional information for pose estimation task. In addition, we give pixels near key point region a higher weight and design a filtering mechanism on the output results, we only back-propagate and update the model parameters using the results above the confidence level. Through experiments, we verify that our model is able to gain information from unlabeled data to improve the accuracy of key point detection and overcome the problem of insufficient data in surgical instrument pose estimation task.