Glottal Gap tracking by a continuous background modeling using inpainting

Abstract

The visual examination of the vibration patterns of the vocal folds is an essential method to understand the phonation process and diagnose voice disorders. However, a detailed analysis of the phonation based on this technique requires a manual or a semi-automatic segmentation of the glottal area, which is difficult and time consuming. The present work presents a cuasi-automatic framework to accurately segment the glottal area introducing several techniques not explored before in the state of the art. The method takes advantage of the possibility of a minimal user intervention for those cases where the automatic computation fails. The presented method shows a reliable delimitation of the glottal gap, achieving an average improvement of 13 and 18% with respect to two other approaches found in the literature, while reducing the error of wrong detection of total closure instants. Additionally, the results suggest that the set of validation guidelines proposed can be used to standardize the criteria of accuracy and efficiency of the segmentation algorithms.

Appendix

In order to assess the glottal segmentation, different metrics have been proposed in the literature: DICE and an area error [21]; a multipoint scale comparison [9]; mean square error [30]; and tracking and comparison of some points of interest [8]. However, it is unreasonable to expect all the metrics be valid for the glottal segmentation problem, since each metric has sensitivities to different properties of the segmentation and thus can discover different types of error [31]. Hence, there is a need for standardizing not only the metrics to be used but also the definition of each metric. In order to overcome this problem, we propose a complete objective framework to evaluate the glottal segmentation.

The initial metrics computed in this study can be categorized depending on their nature and their definition as overlap based, pair-counting based, information theory based, probabilistic based, and spatial distance based. For a detailed description of all the metrics enumerated, next refer to [31].

The first group computes the degree of overlap between two segmentations. To this group belongs DICE or overlap index, Jaccard (JAC), true positive rate (TPR) or sensitivity, true negative rate (TNR) or specificity, F1-Score (FMS), false positive rate (FPR), false negative rate (FNR), positive predictive value or precision (PPV), global constancy error (GCE), and object-level consistency error (OCE).

The second group measures the similarity between clusterings. One of its important properties is that is not based on labels, and thus can be used to evaluate clusterings as well as classifications. The metrics implemented in this work were Rand Index (RI) and Adjusted Rand Index (ARI).

The third group computes a measure of information content for each segmentation. The variation of information (VI) figure of merit belongs to this group. It measures the amount of information that one segmentation shares with the other.

The fourth group describes metrics defined as functions of statistics calculated from the pixels in the overlapped regions of the segmentations. The metrics included are Cohen Kappa coefficient (KAP) and the area under the ROC curve (AUC).

Lastly, the metrics based on spatial distance are defined as functions of the euclidean distances between the pixels that belong to the ground-truth and the pixels of the automatic segmentation. The metrics used in this category are Hausdorff distance (HD), average Hausdorff distance (AHD), and Pratt Index.

However, not all of these 18 metrics were finally considered to objectively evaluate the segmentation accuracy of this study. The properties to be fulfilled by the metrics to be considered part of the good metrics set are the following:

• Contour accuracy: The segmentations have to provide boundary delimitations as exact as possible. The metrics that are more sensitive to point positions, as distance based metrics, are more suitable to evaluate the segmentation.

• Degree of overlapping: The segmentations have to provide a correct location of the segmented object (alignment between segmentation and ground-truth). This aspect is important to rank correctly the instants of total closure. The metrics suitable for this property are the overlap based.

• Complex boundary: The segmentations lead with non-regular shapes, thus the metrics that are sensitive to pixels positions are more suitable to evaluate the final results. The most suitable metrics are the distance-based ones.

• Background dominates: All the metrics based on a true negative factor (pixels correctly detected as background) have to be avoided. Such metrics are biased with respect to the ratio between the number of foreground pixels (glottis) and the number of background pixels (glottal structures), producing a class imbalance when the background represents the largest part, as occurs in the glottal segmentation.

• Over and under-segmentation penalization: The metrics have to penalize equally the over and under-segmentation. Thus, metrics such as VI, FPR, PPV, and TPR have to be avoided.

• High class imbalance: when the segmentation process produces small regions, metrics with chance adjustment, such as KAP and ARI, are recommended.

• Outlier sensitivity: Sometimes, automatic segmentations have outliers in form of a small set of pixels outside of the right target area. The outlier sensitivity describes metrics that penalize such outliers.

In those cases with no unique metric fulfilling all the properties at the same time, a combination of more than one metric will be necessary. Also, a good practice is to reject metrics that have similar definitions to avoid redundant information. Table 6 shows in rows the metrics, and in columns the guidelines that have to be followed. The three last columns summarize the results of the three trials, μ represents the average accuracy of 760 images analyzed, and 𝜖 _close rates how many times an image was ranked with 0. A zero 𝜖 _close can be understood as not overlapping or not segmented images, which is related with the error introduced at the closed instants.

Table 6 Summary of the 18 metrics with the selection guidelines. Each row corresponds to one of the metrics, the first seven columns correspond to the properties evaluated to be part of the good metrics set, and the last three columns are the average values of each metric for the three assessments

Having the aforementioned in mind, Table 6 shows a check (✓) to denote a metric that is recommended for the corresponding property; a cross (X) denotes that the metric is not recommended; and empty cells denote neutrality. The good metrics are the ones that have at least one check without crosses. The metrics that satisfy this statement are highlighted in yellow: DICE, JAC, FMS, ARI, KAP, AHD, and Pratt. In order to avoid redundancy, JAC and FMS were excluded because they provide a similar ranking than the DICE coefficient (JAC and FMS are derived from the DICE equation). Following the same criteria, AHD and Pratt are metrics based on distance errors, so one of them may be excluded. Since AHD does not rank the similarity between segmentations in a range scale (as Pratt does, between 0 and 1), we consider that is less intuitive, and it has also been excluded.

Thus, the metrics that best suit the guidelines are one based on overlapping (DICE), one based on pair-counting (ARI), one based on probabilistic means (KAP), and one based on distance (Pratt).

Lastly, in order to verify the concordance between the metrics, pairwise Pearson’s correlation coefficients were calculated. The 760 ranks obtained for each metric are correlated between them and deployed on Table 7. Results show a great correlation between metrics hence we choose DICE and Pratt as good metrics.

Table 7 Pearson’s correlation coefficients among the good metrics. Correlations correspond to MaN vs. InP, MaN vs. SrG, and MaN vs. SnW trials