Efficient Uncertainty Estimation in Spiking Neural Networks via MC-dropout

Abstract

Spiking neural networks (SNNs) have gained attention as models of sparse and event-driven communication of biological neurons, and as such have shown increasing promise for energy-efficient applications in neuromorphic hardware. As with classical artificial neural networks (ANNs), predictive uncertainties are important for decision making in high-stakes applications, such as autonomous vehicles, medical diagnosis, and high frequency trading. Yet, discussion of uncertainty estimation in SNNs is limited, and approaches for uncertainty estimation in ANNs are not directly applicable to SNNs. Here, we propose an efficient Monte Carlo(MC)-dropout based approach for uncertainty estimation in SNNs. Our approach exploits the time-step mechanism of SNNs to enable MC-dropout in a computationally efficient manner, without introducing significant overheads during training and inference while demonstrating high accuracy and uncertainty quality.

Appendix

Proper Scoring Rules. A scoring rule $S(\textbf{p},y)$ assigns a value for a predictive distribution $\textbf{p}$ and one of the labels y. A scoring function $s(\textbf{p},\textbf{q})$ is defined as the expected score of $S(\textbf{p},y)$ under the distribution $\textbf{q}$

$$\begin{aligned} s(\textbf{p},\textbf{q}) = \sum _{y=1}^{K} q_yS(\textbf{p}, y). \end{aligned}$$

(8)

If a scoring rule satisfies $s(\textbf{p},\textbf{q}) <= s(\textbf{q},\textbf{q})$, it is called a proper scoring rule. If $s(\textbf{p},\textbf{q}) = s(\textbf{q},\textbf{q})$ implies $\textbf{q}=\textbf{p}$, this scoring rule is a strictly proper scoring rule. When evaluating quality of probabilities, an optimal score output by a proper scoring rule indicates a perfect prediction [17]. In contrast, trivial solutions could generate optimal values for an improper scoring rule [8, 17].

The two most commonly used proper scoring rules are Brier score [1] and NLL. Brier score is the squared $L_2$ norm of the difference between $\textbf{p}$ and one-hot encoding of the true label y. NLL is defined as $S(\textbf{p}, y) = -\textrm{log} p(y\vert \textbf{x})$ with y being the true label of the sample $\textbf{x}$. Among these two rules, the Brier score is more recommendable because NLL can unacceptably over-emphasize small differences between small probabilities [17]. Note that proper scoring rules are often used as loss functions to train neural networks. [8, 13].

ECE. The ECE is a scalar summary statistic of calibration that approximates miscalibration [10, 15]. To calculate ECE, the predicted probabilities,

$\hat{y}_n = \textrm{argmax}_y \textbf{p}(y\vert \mathbf {x_n})$, of test instances are grouped into M equal-interval bins. The ECE is defined as

$$\begin{aligned} ECE = \sum _{m=1}^M f_m \vert o_m - e_m\vert , \end{aligned}$$

(9)

where $o_m$ is the fraction of corrected classified instances in the $m^{th}$ bin, $e_m$ the average of all the predicted probabilities in the $m^{th}$ bin, and $f_m$ the fraction of all the test instances falling into the $m^{th}$ bin. The ECE is not a proper scoring rule and thus optimum ECEs could come from trivial solutions.