Rethinking Label Smoothing on Multi-Hop Question Answering

Abstract

Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction. In this work, we present the first application of label smoothing to the MHQA task, aiming to enhance generalization capabilities in MHQA systems while mitigating overfitting of answer spans and reasoning paths in the training set. We introduce a novel label smoothing technique, F1 Smoothing, which incorporates uncertainty into the learning process and is specifically tailored for Machine Reading Comprehension (MRC) tasks. Moreover, we employ a Linear Decay Label Smoothing Algorithm (LDLA) in conjunction with curriculum learning to progressively reduce uncertainty throughout the training process. Experiment on the HotpotQA dataset confirms the effectiveness of our approach in improving generalization and achieving significant improvements, leading to new state-of-the-art performance on the HotpotQA leaderboard.

Y. Wang—Equal contribution.

7 Appendix A

In order to alleviate the complexity introduced by multiple for loops in the F1 Smoothing method, we have optimized Eq. (12) and Eq. (13). We use $L_a=e^{*}-s^{*}+1$ and $L_p=e-s+1$ to denote respectively the length of gold answer and predicted answer.

$$\begin{aligned} q_s(t|x)=\sum _{\xi =t}^{L-1} \text {F1}\left( (t,\xi ),a_{\text {gold}}\right) . \end{aligned}$$

(16)

If $t < s^{*}$, the distribution is

$$\begin{aligned} q_s(t|x)= \sum _{\xi =s^{*}}^{e^{*}} \frac{2(\xi -s^{*}+1)}{L_p+L_a} + \sum _{\xi =e^{*}+1}^{L-1} \frac{2L_a}{L_p+L_a}, \end{aligned}$$

(17)

else if $s^{*} \le t \le e^{*}$, we have the following distribution

$$\begin{aligned} q_s(t|x)=\sum _{\xi =s}^{e^{*}} \frac{2L_p}{L_p+L_a} + \sum _{\xi =e^{*}+1}^{L-1} \frac{2(e^{*}-s+1)}{L_p+L_a}. \end{aligned}$$

(18)

In Eq. 17 and 18, $L_p=e-i+1$.

We can get $q_e(t|x)$ similarly. If $t > e^{*}$,

$$\begin{aligned} q_e(t|x)= \sum _{\xi =s^{*}}^{e^{*}} \frac{2(e^{*}-\xi +1)}{L_p+L_a} + \sum _{\xi =0}^{s^{*}-1} \frac{2L_a}{L_p+L_a}, \end{aligned}$$

(19)

else if $s^{*} \le t \le e^{*}$,

$$\begin{aligned} q_e(t|x)= \sum _{\xi =s^{*}}^{e} \frac{2L_p}{L_p+L_a} + \sum _{\xi =0}^{s^{*}-1} \frac{2(e-s^{*}+1)}{L_p+L_a}. \end{aligned}$$

(20)

In Eqs. 19 and 20, $L_p=i-s+1$.