What is Cromwell’s Rule?
Cromwell’s rule states that no prior probability distribution should be 100% true or false, since accuracy requires allowing for unknowable “X factors.” With the exception of logic statements that need a definitive true or false decision, such as 1+1=2 or If than And statements, there should not be complete confidence of an outcome when assigning initial probabilities to a model. This rule is applied to both Bayesian and Frequentist approaches by ensuring that the prior probability distribution for any outcome does not equal 0 (will never happen) or 1 (guaranteed to happen).
In Bayesian statistics and inference, if the prior probability is a “hard coded” definitive yes or no answer, then no amount of evidence to the contrary will change the posterior probability distribution, thereby corrupting the model’s outcome.
In Frequentist statistics, this approach is used to account for the remote possibilities that new observations will vary from the long-run frequency. For example, flipping a coin should not guarantee either a heads or tails outcome, but rather assign a small chance that the coin lands and balances on its edge.