Model change and reliability in scientific inference

Abstract

One persistent challenge in scientific practice is that the structure of the world can be unstable: changes in the broader context can alter which model of a phenomenon is preferred, all without any overt signal. Scientific discovery becomes much harder when we have a moving target, and the resulting incorrect understandings of relationships in the world can have significant real-world and practical consequences. In this paper, we argue that it is common (in certain sciences) to have changes of context that lead to changes in the relationships under study, but that standard normative accounts of scientific inquiry have assumed away this problem. At the same time, we show that inference and discovery methods can “protect” themselves in various ways against this possibility by using methods with the novel methodological virtue of “diligence.” Unfortunately, this desirable virtue provably is incompatible with other desirable methodological virtues that are central to reliable inquiry. No scientific method can provide every virtue that we might want.

Appendices

Appendix 1: Notation

Let $X$ represent a random sequence of data. Let $X_B^t$ represent a random subsequence of length $t$ of data generated from distribution $B$. Let $\mathbf{F}$ be a framework (in this case, a set of distributions). Let $M_\mathbf{F}$ be a method that takes a data sequence $X$ as input and outputs a distribution $B \in \mathbf{F}$; we will typically drop the subscript $\mathbf{F}$ from $M$ as we will be dealing with a single framework at a time. Concretely, $M[X_B^t]=O$ means that $M$ outputs $O$ after observing the sequence $X_B^t$. Let $D$ be a distance metric over distributions (e.g., the Anderson-Darling test). Let $D_\delta (A,B)$ be shorthand for the following inequality: $D(A,B)<\delta$. Finally, let $[X,Y]$ denote the concatenation of sequence $X$ with sequence $Y$.

Definition

A distribution $A$ is absolutely continuous with respect to another distribution $B$ iff $\forall x\ P_{B}(x)=0 \Rightarrow P_{A}(x)=0$. That is, if $B$ gives probability $0$ to some event $x$, then $A$ also gives probability $0$ to that same event. Let $AC(A)$ be the set of distributions which are absolutely continuous with respect to $A$ except for $A$ itself.

Definition

An estimator $M$ is consistent if $\forall B\in \mathbf{F}\ \forall \delta >0 \lim _{n\rightarrow \infty } P(D_\delta (M[X_B^t],B))\rightarrow 1$. That is, for all distributions in the framework, the probability that $M$’s output is arbitrarily close to the target distribution approaches 1 as the amount of data increases to infinity.

Definition

An estimator $M$ can be forced to make arbitrary errors if $\forall B_1\in \mathbf{F}\ \forall B_2\in AC(B_1)\cap \mathbf{F}\ \forall \delta , \epsilon >0\ \forall n_2 \exists n_1 P(D_\delta (M[X_{B_1}^{n_1},X_{B_2}^{n_2}],B_2))\le \epsilon$. That is, consider any distribution $B_2$ which is in the framework and is absolutely continuous with respect to $B_1$. Then for any amount of data $n_2$ from $B_2$, there is an amount of data $n_1$ from $B_1$ such that $M$’s output will still be arbitrarily unlikely to be arbitrarily close to $B_2$ after seeing the $n_1 + n_2$ data.

Appendix 2: Lemma: consistency $\Rightarrow$ arbitrary errors (within AC)

Proof

Assume $M$ is consistent. It suffices to show that:

$$\begin{aligned} \forall B_1 \in \mathbf{F}\ \forall n_2 >0\ \forall B_2\in AC(B_1)\cap \mathbf{F}\ \forall \delta >0\ \forall \epsilon <1\ \exists n_1 P(D_\delta (M[X_{B_1}^{n_1},X_{B_2}^{n_2}],B_1))>\epsilon \end{aligned}$$

That is, even if we add a finite sequence of data drawn from $B_2$ to the end of any $X_{B_1}^{n_1}$ sequence, then there is some amount of $B_1$ data so that the estimator $M$ still converges to $B_1$.

Choose arbitrary $B_1$, $B_2$ and $n_2$. Let $S$ be the set of all events in the metric space that, if satisfied by $X_{B_2}^{n_2}$, would stop $M$ from converging to $B_1$. That is, let $S$ be the set of all events that, if satisfied by $X_{B_2}^{n_2}$, would entail the negation of:

$$\begin{aligned} \forall \delta >0\ \forall \epsilon <1 \ \exists n_1P(D_\delta (M[X_{B_1}^{n_1},X_{B_2}^{n_2}],B_1))>\epsilon \end{aligned}$$

Since $M$ is consistent for $B_1$, then $P(X_{B_1}^{n_1}\in S)=0$. Since $B_2$ is absolutely continuous with respect to $B_1$, $P(X_{B_2}^{n_2}\in S)=0$. As such, it is at most a probability $0$ event that $X_{B_2}^{n_2}$ can take a value that prevents $M$ from converging to $B_1$, so $M$ will still converge in probability to $B_1$ over sequences of the form $[X_{B_1}^{n_1},X_{B_2}^{n_2}]$. $\square$

Appendix 3: Construction: diligence $\Rightarrow \lnot$ arbitrary errors

We construct the formal definition of diligence from that of “arbitrary errors” (AE) in a way that makes it clear that diligent methods are not subject to arbitrary errors. The negation of AE is:

$$\begin{aligned} \exists B_1 \in \mathbf{F}\ \exists B_2\in AC(B_1)\cap \mathbf{F}\ \exists \delta >0\ \exists \epsilon <1\ \exists n_2 \forall n_1 P(D_\delta (M[X_{B_1}^{n_1},X_{B_2}^{n_2}],B_2))> \epsilon \end{aligned}$$

This condition is, however, insufficiently weak to capture diligence, as we want to avoid such errors for all pairs of distributions in the framework, not just for some absolutely continuous pair. We thus strengthen the negation of AE by turning the two leading existential quantifiers into universal quantifiers and extending the domain of the universal quantifier over $B_2$ to include those distributions which are not absolutely continuous with respect to $B_1$:

Definition

An estimator $M$ is diligent if

$$\begin{aligned} \forall B_1\in \mathbf{F}\ \forall B_2\in \mathbf{F}\backslash B_1\ \forall \delta >0\ \exists \epsilon >0\ \exists n_2 \forall n_1 P(D_\delta (M[X_{B_1}^{n_1}, X_{B_2}^{n_2}], B_2)) > \epsilon . \end{aligned}$$

That is, for any pair of distributions in the framework, there is an amount of data $n_2$ from $B_2$ such that $M$’s output will be arbitrarily close to $B_2$ with positive probability after seeing $n_1 + n_2$ data, for any amount of data $n_1$ from $B_1$.

Definition

A framework $\mathbf{F}$ is nontrivial iff there exists some $B \in \mathbf{F}$ such that $AC(B)\cap \mathbf{F} \ne \emptyset$.

Clearly, diligence implies the negation of AE for all nontrivial frameworks. We thus have the key theorem for this paper:

Theorem

No statistical estimator for a (nontrivial) framework is both consistent and diligent.

Proof

Assume $M$ is both consistent and diligent. Its consistency implies that AE holds for it. Its diligence, along with the nontriviality of the framework, implies that $\lnot$AE holds for it. Contradiction, and so no $M$ can be both consistent and diligent for a nontrivial framework. $\square$

Appendix 4: Generalizing diligence

A natural generalization of diligence yields a novel methodological virtue: Uniform Diligence. Uniform diligence is a strengthening of regular (pointwise) diligence in the same way that uniform consistency is a strengthening of pointwise consistency. Instead of requiring only that, for each $B_1, B_2$ and $\delta$, there be some $n_2$, Uniform Diligence requires that there be some $n_2$ which works for all such combinations.

Definition

An estimator $M$ is uniformly diligent if

$$\begin{aligned} \exists n_2 \forall B_1\in \mathbf{F}\ \forall B_2\in \mathbf{F}\backslash B_1\ \forall \delta >0\ \exists \epsilon >0\ \forall n_1 P(D_\delta (M[X_{B_1}^{n_1},X_{B_2}^{n_2}],B_2))> \epsilon . \end{aligned}$$

Obviously, consistency and uniform diligence are also incompatible, as the latter is a strengthening of diligence. The following chart shows three different ways of ordering the quantifiers in the definition of Diligence, producing methodological virtues of varying strength. The weakest, Responsiveness, is not incompatible with consistency. For space and clarity, B is used in place of $\forall B_1\in \mathbf{F}\ \forall B_2\in \mathbf{F}\backslash B_1\ \forall \delta >0\ \exists \epsilon >0$.

Responsiveness	Diligence	Uniform diligence
$\mathbf{B} \forall n_1 \exists n_2$	$\mathbf{B} \exists n_2 \forall n_1$	$\exists n_2 \mathbf{B} \forall n_1$