Gambling Adversarial Nets for Hard Sample Mining and Structured Prediction: Application in Ultrasound Thyroid Nodule Segmentation

Abstract

Most real-world datasets are characterized by long-tail distributions over classes or, more generally, over underlying visual representations. Consequently, not all samples contribute equally to the training of a model and therefore, methods properly evaluating the importance/difficulty of the samples can considerably improve the training efficiency and effectivity. Moreover, preserving certain inter-pixel/voxel structural qualities and consistencies in the dense predictions of semantic segmentation models is often highly desirable; accordingly, a recent trend of using adversarial training is clearly observable in the literature that aims for achieving higher-level structural qualities. However, as we argue and show, the common formulation of adversarial training for semantic segmentation is ill-posed, sub-optimal, and may result in side-effects, such as the disability to express uncertainties.

In this paper, we suggest using recently introduced Gambling Adversarial Networks that revise the conventional adversarial training for semantic segmentation, by reformulating the fake/real discrimination task into a correct/wrong distinction. This forms then a more effective training strategy that simultaneously serves for both hard sample mining as well as structured prediction. Applying the gambling networks to the ultrasound thyroid nodule segmentation task, the new adversarial training dynamics consistently improve the qualities of the predictions shown over different state-of-the-art semantic segmentation architectures and various metrics.