The Full Bayesian Significance Test (FBST) is a novel statistical test of hypothesis published in 1999 by both authors  and further extended in [72, 90]. This solution is anchored by a novel measure of statistical significance known as the -value, , a.k.a. the evidence value provided by observational data in support of the statistical hypothesis or, the other way around, the epistemic value of hypothesis given the observational data . The -value, its theoretical properties and its applications have been a topic of research for the Bayesian Group at USP, the University of São Paulo, for the last 20 years, including collaborators working at UNICAMP, the State University of Campinas, UFSCar, the Federal University of São Carlos, and other universities in Brazil and around the world. The bibliographic references list a selection of contributions to the FBST research program and its applications.
The FBST was specially designed to provide a significance measure to sharp or precise statistical hypothesis, namely, hypotheses consisting of a zero-volume (or zero Lebesgue measure) subset of the parameter space. Furthermore the e-value has many necessary or desirable properties for a statistical support function, such as:
(i) Give an intuitive and simple measure of significance for the hypothesis in test, ideally, a probability defined directly in the original or natural parameter space.
(ii) Have an intrinsically geometric definition, independent of any non-geometric aspect, like the particular parameterization of the (manifold representing the) null hypothesis being tested, or the particular coordinate system chosen for the parameter space, in short, be defined as aninvariant procedure.
(iii) Give a measure of significance that is smooth, i.e. continuous and differentiable, on the hypothesis parameters and sample statistics, under appropriate regularity conditions for the model.
(v) Require no ad hoc artifice
like assigning a positive prior probability to zero measure sets, or setting an arbitrary initial belief ratio between hypotheses.
(vi) Be a possibilistic support function, where the support of a logical disjunction is the maximum support among the support of the disjuncts, see .
(vii) Be able to provide a consistent test for a given sharp hypothesis.
(viii) Be able to provide compositionality operations in complex models.
(ix) Be an exact procedure, i.e., make no use of “large sample” asymptotic approximations when computing the -value.
(x) Allow the incorporation of previous experience or expert’s opinion via (subjective) prior distributions.
The objective of the next two sections is to recall standard nomenclature and provide a short survey of the FBST theoretical framework, summarizing the most important statistical properties of its statistical significance measure, the -value; these introductory sections follow closely the tutorial [115, appendix A], see also .
2 Bayesian Statistical Models
In frequentist or classical statistics, one is allowed to use probability calculus in the sample space, but strictly forbidden to do so in the parameter space, that is, is to be considered as a random variable, while
is not to be regarded as random in any way. In frequentist statistics,should be taken as a “fixed but unknown quantity”, and neither probability nor any other belief calculus may be used to directly represent or handle the uncertain knowledge about the parameter.
In the Bayesian context, the parameter
is regarded as a latent (non-observed) random variable. Hence, the same formalism used to express (un)certainty or belief, namely, probability theory, is used in both the sample and the parameter space. Accordingly, the joint probability distribution,
should summarize all the information available in a statistical model. Following the rules of probability calculus, the model’s joint distribution ofand can be factorized either as the likelihood function of the parameter given the observation times the prior distribution on , or as the posterior density of the parameter times the observation’s marginal density,
The prior probability distribution represents the initial information available about the parameter. In this setting, a predictive distribution for the observed random variable, , is represented by a mixture (or superposition) of stochastic processes, all of them with the functional form of the sampling distribution,according to the prior mixing (or weight) distribution,
If we now observe a single event, , it follows from the factorizations of the joint distribution above that the posterior probability distribution of , representing the available information about the parameter after the observation, is given by
In order to replace the ‘proportional to’ symbol, , by an equality, it is necessary to divide the right hand side by the normalization constant, . This is the Bayes rule, giving the (inverse) probability of the parameter given the data. That is the basic learning mechanism of Bayesian statistics. Computing normalization constants is often difficult or cumbersome. Hence, especially in large models, it is customary to work with unormalized densities or potentials as long as possible in the intermediate calculations, computing only the final normalization constants. It is interesting to observe that the joint distribution function, taken with fixed and free argument , is a potential for the posterior distribution.
Bayesian learning is a recursive process, where the posterior distribution after a learning step becomes the prior distribution for the next step. Assuming that the observations are i.i.d. (independent and identically distributed) the posterior distribution after observations, , becomes,
If possible, it is very convenient to use a conjugate prior
, that is, a mixing distribution whose functional form is invariant by the Bayes operation in the statistical model at hand. For example, the conjugate priors for the Normal and Multivariate models are, respectively, Wishart and the Dirichlet distributions, see[42, 138].
The founding fathers of the Bayesian school, namely, Reverend Thomas Bayes, Richard Price and Pierre-Simon de Laplace, interpreted the Bayesian operation as a path taken for learning about probabilities related to unobservable causes, represented by the parameters of a statistical model, from probabilities related to their consequences, represented by observed data. Nevertheless, later interpretations of statistical inference, like those of Bruno de Finetti who endorsed the epistemological perspectives of empirical positivism, strongly discouraged such causal interpretations, see [121, 122] for further discussion of this controversy.
The ‘beginnings and the endings’ of the Bayesian learning process deserve further discussion, that is, we should present some rationale for choosing the prior distribution used to start the learning process, and some convergence theorems for the posterior as the number observations increases. In order to do so, we must access and measure the information content of a (posterior) distribution. [52, 56, 116, 138] explain how the concept of entropy can be used to unlock many of the mysteries related to the problems at hand. In particular, they discuss some fine details about criteria for prior selection and important properties of posterior convergence.
3 The Epistemic -values
Let be a vector parameter of interest, and be the likelihood associated to the observed data , as in the standard statistical model. Under the Bayesian paradigm the posterior density, , is proportional to the product of the likelihood and a prior density,
A hypothesis states that the parameter lies in the null set, defined by inequality and equality constraints given by vector functions and in the parameter space,
From now on, we use a relaxed notation, writing instead of . We are particularly interested in sharp (precise) hypotheses, i.e., those in which there is at least one equality constraint and, therefore, .
The FBST defines , the -value supporting (in favor of) the hypothesis , and , the -value against , as
The function is known as the posterior surprise function relative to a given reference density, . is the cumulative surprise distribution. Due to its interpretation in mathematical and philosophical logic, see , is also known as (the statistical model’s) truth function or Wahrheitsfunktion. The surprise function was used in the context of statistical inference by Good , Evans , Royall  and Schackle [102, 103], among others. Its role in the FBST is to make explicitly invariant under suitable transformations on the coordinate system of the parameter space, see next section.
The tangential (to the hypothesis) set , is a Highest Relative Surprise Set (HRSS). It contains the points of the parameter space with higher surprise, relative to the reference density, than any point in the null set . When , the possibly improper uniform density, is the Posterior’s Highest Density Probability Set (HDPS) tangential to the null set . Small values of indicate that the hypothesis traverses high density regions, favoring the hypothesis.
Notice that, in the FBST definition, there is an optimization step and an integration step. The optimization step follows a typical maximum probability argument, according to which, “a system is best represented by its highest probability realization”. The integration step extracts information from the system as a probability weighted average. Many inference procedures of classical statistics rely basically on maximization operations, while many inference procedures of Bayesian statistics rely on integration (or marginalization) operations. In order to achieve all its desired properies, the FBST procedure has to use both operation types.
3.1 Nuisance Parameters
Let us consider the situation where the hypothesis constraint, is not a function of some of the parameters, . This situation is described in  by Debabrata Basu as follows:
“If the inference problem at hand relates only to , and if information gained on
is of no direct relevance to the problem, then we classifyas the Nuisance Parameter. The big question in statistics is: How can we eliminate the nuisance parameter from the argument?”
Basu goes on listing at least 10 categories of procedures to achieve this goal, like using or , the maximization or integration operators, in order to obtain a projected profile or marginal posterior function, . The FBST does not follow the nuisance parameters elimination paradigm, working in the original parameter space, in its full dimension.
3.2 Reference Prior and Invariance
In the FBST the role of the reference density, is to make explicitly invariant under suitable transformations of the coordinate system. The natural choice of reference density is an uninformative prior, interpreted as a representation of no information in the parameter space, or the limit prior for no observations, or the neutral ground state for the Bayesian learning operation. Standard (possibly improper) uninformative priors include the uniform, maximum entropy densities, or Jeffreys’ invariant prior. Finally, invariance, as used in statistics, is a metric concept, and the reference density can be interpreted as induced by the statistical model’s information metric in the parameter space, , see [1, 10, 16, 35, 42, 52, 56, 57, 138] for a detailed discussion. Jeffreys’ invariant prior is proportional to the square root of the information matrix determinant, .
Proof of invariance:
Consider a proper (bijective, integrable, and almost surely continuously differentiable) reparameterization . Under the reparameterization, the Jacobian, surprise, posterior and reference functions are:
Let . It follows that
hence, the tangential set, , and
3.3 Asymptotics and Consistency
Let us consider the cumulative distribution of the evidence value against the hypothesis, , given , the true value of the parameter. Under appropriate regularity conditions, for increasing sample size, , we can say the following:
- If is false, , then converges (in probability) to 1, that is, .
- If is true, , then , the confidence level, is approximated by the function
is the cumulative chi-square distribution withdegrees of freedom.
Proof of consistency:
Let be the cumulative distribution of the evidence value against the hypothesis, given . We stated that, under appropriate regularity conditions, for increasing sample size, , if is true, i.e. , then , is approximated by the function
Let , and be the true value, the unconstrained MAP (Maximum A Posteriori), and constrained (to
) MAP estimators of the parameter.
Since the FBST is invariant, we can chose a coordinate system where, the (likelihood function) Fisher information matrix at the true parameter value is the identity, i.e., . From the posterior Normal approximation theorem, see , we know that the standarized total difference between and
converges in distribution to a standard Normal distribution, i.e.
This standarized total difference can be decomposed into tangent (to the hypothesis manifold) and transversal orthogonal components, i.e.
Hence, the total, tangent and transversal distances ( norms), , and , converge in distribution to chi-square variates with, respectively, , and degrees of freedom.
Also from, the MAP consistency, we know that the MAP estimate of the Fisher information matrix, , converges in probability to true value, .
Now, if converges in distribution to , and converges in probability to , we know that the pair converges in distribution to . Hence, the pair converges in distribution to , where is a chi-square variate with degrees of freedom. So, from the continuous mapping theorem, the evidence value against , , converges in distribution to , where is a chi-square variate with degrees of freedom.
Since the cumulative chi-square distribution is an increasing function, we can invert the last formula, i.e., . But, since in a chi-square variate with degrees of freedom,
A similar argument, using a non-central chi-square distribution, proves the other asymptotic statement.
If a random variable,
, has a continuous and increasing cumulative distribution function,, the random variable
has uniform distribution. Hence, the tranformation, defines a “standarized -value”, , that can be used somewhat in the same way as a -value of classical statistics. This standarized -value may be a convenient form to report, since its asymptotically uniform distribution (under ) provides a large-sample limit interpretation, and many researchers will feel already familiar with consequent diagnostic procedures for scientific hypotheses based on adequately large empirical data-sets.
4 A Survey of FBST Related Literature
A systematic cataloging of all published articles related to this research program is beyond the scope of this article; in the next subsections we survey a selection of such articles. The selected articles provides a sample covering diverse areas like statistical theory and methods, applications to statistical modeling and operations research, and research in foundations of statistics, logic and epistemology. This selection is certainly biased, favoring the the authors’ personal taste or involvement.
4.1 Statistical Theory
Several authors have developed the statistical theory that provides the mathematical formalism and demonstrates the outstanding statistical properties FBST and its significance measure, the -value. The following articles have explored and developed these themes of research:
 is the first article of this research program. It presents the basic definition of the -value and the FBST, and gives several simple and intuitive applications.  provides an explicitly invariant version of the inference procedures. After a long process in which the authors had to overcome objections raised by influential mainstream Bayesian thinkers,  was published in the flagship journal of ISBA - the International Society for Bayesian Analysis.  provides an entry on the FBST in the International Encyclopedia of Statistical Science.
[18, 92, 93, 99, 130, 131, 132] develop higher order asymptotic approximations of (log) likelihood and pseudo-likelihood functions that, in turn, are used do develop high-precision but fast computational algorithms for calculating
-values in parametric models. The availability of a good library of such fast and reliable computer programs will, in turn, we believe, facilitate the incorporation of the FBST in statistical softwares intended for end-users or routine applications.
4.2 Statistical Modeling
Several authors have developed a wide range of applications of the FBST to statistical modeling and operations research. The following articles have explored and developed these themes of research:
 applies the FBST to software compliance testing and certification.
 applies the FBST to detect equilibrium conditions, or the lack thereof, in market prices of economic commodities or financial derivative contracts.
 applies the FBST in the context of empirical economic studies.
 use the FBST for selection and testing of statistical copulas.
 applies the FBST to model selection in statistical studies conducted under informative sampling conditions.
4.3 Foundations of Statistics, Logic and Epistemology
Traditional significance measures used in statistics are always designed to work in tandem with a specific epistemological framework that gives them an appropriate interpretive context and support. For example,
-values are usually presented in the context of the “judgment metaphor” and the deductive falibilism epistemological framework, as developed by the philosopher Karl Popper, among others. Meanwhile, Bayes factors are presented in the context of the “gambling metaphor” and utility based decision theory, as developed by Bruno de Finetti, see[31, 36, 53, 54]. Furthermore, the logic or algebraic properties of each significance measure, in its appropriate domain of statistical hypotheses, must be mutually supportive and compatible with intended interpretations. The following articles have explored and developed these themes of research:
[72, 126] compare the theoretical properties of the -value with those of traditional significance measures, like the -value and Bayes Factors. These articles analyze in great detail historical arguments given by celebrated statisticians against the use of procedures based on highest density probability sets. Among those that opposed such ideas is Dennis Lindley, an influential figure at IME-USP and a personal friend of the first author. Finally, [72, 126] analyze historical desiderata for an acceptable Bayesian significance test that were formulated by the frequentist statistician Oscar Kempthorne to the first author, and show how the FBST successfully achieves all these desired characteristics.
 analyzes the composition of hypotheses defined in independent statistical models and the corresponding composition rules for -values and truth functions.
 studies significance measures for evidence amalgamation and meta-analysis.
[33, 38, 51, 32, 124] analyze conditions of logical consistency for significance measures and test procedures for several hypotheses defined in the same statistical model. Conversely, these articles fully characterize some (agnostic or trivalent) generalizations of the FBST as the only statistical tests satisfying such logical consistency conditions.
 analyzes solutions to the problem of (statistical) induction, including Bayesian perspectives in general and the FBST in particular.
[121, 123] analyze the philosophical premises used by Karl Pearson to define the -value and to establish the epistemological foundations of frequentist statistics; why Pearson’s work and the subsequent work of Bruno de Finetti reversed previous commitments of Bayesian statistics; and how the FBST can be seen a way to reenter the path envisioned by the founding fathers of the Bayesian school, namely, Reverend Thomas Bayes, Richard Price and Pierre-Simon de Laplace.
5 Future Research and Final Remarks
The FBST research program has grown and spread far and wide, in some directions suggested by these authors, and also in other directions that were for us completely unforeseen and wonderfully surprising. We are confident that this research program will continue to flourish and expand, exploring new areas of theory and application. The authors would like to suggest a few topics (focusing on theoretical and applied statistics) worthy of further attention as possible entry points for those interested (be all welcome) in joining this research program:
(1) In the context of information based medicine, see 
, it is important to compare and test the sensibility and specificity of alternative diagnostic tools, access the bio-equivalence of drugs coming from different suppliers, identify and test the efficacy of possible genetic markers for clinical conditions, etc. How to combine fast and computationally inexpensive heuristic algorithms and reliable statistical test procedures to best handle these and similar problems?
(2a) Influence diagrams are a powerful tool for decision modeling, see [7, 86]. Nevertheless, it is often hard to select optimal diagrams to model complex applications, see for example [73, 104]. How can the FBST best be used for sequential or concomitant inclusion/ exclusion of links or edge selection in influence graphs?
(2b) The aforementioned questions also arise in the context of Bayesian networks. In this context, it is important not only to test the significance of individual edges, but also to test the integrity of higher level sparsity structures, like the network click structure or its block factors, see[109, 114, 125, 127].
(3) The -value and the FBST were originally developed for parametric models. How can the -value be used, interpreted, computed (and maybe generalized) in semi-parametric or non-parametric settings? For instance, in models using functional bases, how can we test speeds of convergence for series expansions?
(4a) The compositionality rules established in  are based on functional operations over the truth functions, . [6, 58] present similar rules (for serial-parallel composition) in the context of reliability theory. Can these theories be seen as particular cases of more general and abstract logical formalisms?
(5) The conditions for pragmatic acceptance of sharp hypotheses stated in  depend on consensual bounds for background uncertainties. For universal physical constants, metrologists establish such bounds by aggregating results of diverse experiments; similar situations occur in meta-analysis studies. Several statistical methods have been proposed to aggregate such diverse data-sets, see [25, 40, 59, 60, 77]. What are the best ways to coherently establish and represent aggregate uncertainty bounds in the FBST framework?
Acknowledgments: The authors are grateful to IME-USP - the Institute of Mathematics and Statistics of the University of São Paulo, and INMA-UFMS - the Institute of Mathematics of the Federal University of Mato Grosso do Sul. The authors are extremely grateful for the support received from their colleagues, collaborators, users and critics in the construction works of this research project.
Funding: This research was funded by CNPq - the Brazilian National Counsel of Technological and Scientific Development (grants PQ 302767/2017-7, PQ 301892/ 2015-6); and FAPESP - the State of São Paulo Research Foundation (grants CEPID Shell-RCGI 2014/ 50279-4, CEPID CeMEAI 2013/07375-0).
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in this study’s data analyses, in its methodological developments or conclusions, in writing this manuscript, or in deciding where to publish.
Author Contributions: All authors contributed equally to this paper.
-  Amari, Shun Ichi. Methods of Information Geometry. American Mathematical Society, 2007.
-  Andrade, Plinio; Rifo, Laura Leticia Ramos; Torres, Soledad; Torres-Avilés, Francisco. Bayesian Inference on the Memory Parameter for Gamma-Modulated Regression Models. Entropy 17, 10, 6576-6597, 2015.
-  Andrade, Pablo De Morais; Stern, Julio Michael; Pereira, Carlos Alberto De Bragança. Bayesian Test of Significance for Conditional Independence: The Multinomial Model. Entropy, 16, 3, 1376-1395, 2014.
-  Assane, Cachimo Combo; Pereira, Basilio de Bragança; Pereira, Carlos Alberto de Bragança. Bayesian significance test for discriminating between survival distributions. Communications in Statistics - Theory and Methods, 47, 24, 6095-6107, 2018.
-  Assane, Cachimo Combo; Pereira, Basilio de Bragança; Pereira, Carlos Alberto de Bragança. Model choice in separate families: A comparison between the FBST and the Cox test. Communications in Statistics - Simulation and Computation, 48, 9, 2641-2654, 2019.
-  Barlow, Richard E.; Prochan, Frank. Statistical Theory of Reliability and Life Testing Probability. Models. Silver Spring, To Begin With, 1981.
-  Barlow, Richard E.; Pereira, Carlos Alberto de Bragança. Influence Diagrams and Decision Modelling. p.87-99 in Barlow, Richard E.; Clarotti, Carlo A.; Spizzichino, Fabio (eds.). Reliability and Decision Making. Dordrecht, Springer, 1993.
Barahona, Manuel; Rifo, Laura; Sepúlveda, Maritza; Torres, Soledad. A Simulation-Based Study on Bayesian Estimators for the Skew Brownian Motion.Entropy, 18, 7, 241, 1-14, 2016.
-  Basu, Debabrata; Ghosh, J.K. Statistical Information and Likelihood. Lecture Notes in Statistics, 45, 1988.
-  Bernardo, José M. Reference Analysis. p.17-90 in Dey, D.K.; Rao, C.R. Bayesian Thinking: Modeling and Computation. Handbook of Statistics, v.25. Amsterdam, Elsevier, 2005.
-  Berger, James O.; Wolpert, Rober L. The Likelihood Principle, 2nd ed. Hayward, CA, Inst of Mathematical Statistic, 1988.
-  Bernardini, Diego F. de; Rifo, Laura Leticia Ramos. Full Bayesian significance test for extremal distributions. Journal of Applied Statistics, 38, 4, 851-863, 2011.
-  Bernardo, Gustavo; Lauretto, Marcelo de Souza; Stern, Julio Michael. The full Bayesian significance test for symmetry in contingency tables. AIP Conference Proceedings, 1443, 198-205, 2012.
-  Bonassi, Fernando Vieira; Nishimura, Raphael; Stern, Rafael Bassi. In Defense of Randomization: a Subjectivist Bayesian Approach. AIP Conference Proceedings, 1193, 32-39, 2009;
-  Borges, Wagner; Stern, Julio Michael. The Rules of Logic Composition for the Bayesian Epistemic E-Values. Logic Journal of the IGPL, 15, 5/6, 401-420, 2007.
-  Box, George Edward Pelham; and Tiao, George C. Bayesian Inference in Statistical Analysis. London, Addison-Wesley, 1973.
-  Brentani, Helena; Nakano, Eduardo Y.; Martins, Camila B.; Izbicki, Rafael; Pereira, Carlos Alberto. Disequilibrium Coefficient: A Bayesian Perspective. Statistical Applications in Genetics and Molecular Biology, 10, 1, 22, 1-24, 2011.
-  Cabras, Stefano; Racugno, Walter; Ventura, Laura. Higher order asymptotic computation of Bayesian significance tests for precise null hypotheses in the presence of nuisance parameters. Journal of Statistical Computation and Simulation, 85, 15, 2989-3001, 2015.
-  Cantinha, Rebeca S.; Borrely, Sueli I.; Oguiura, Nancy; Pereira, Carlos A. B.; Rigolona, Marcela M.; Nakano, Eliana. HSP70 expression in Biomphalaria glabrata snails exposed to cadmium. Ecotoxicology and Environmental Safety, 140, 18-23, 2017.
-  Camargo, André P.; Stern, Julio M.; Lauretto, Marcelo S. Estimation and model selection in Dirichlet regression. AIP Conference Proceedings, 1443, 206-213 2012.
-  Cerezetti, Fernando Valvano; Stern, Julio Michael. Non-arbitrage in Financial Markets: A Bayesian Approach for Verification. AIP Conference Proceedings, 1490, 87-96, 2012.
-  Chakrabarty, Dalia. A New Bayesian Test to Test for the Intractability-Countering Hypothesis. Journal of the American Statistical Association, 112, 518, 561-577, 2017.
-  Chen, C. W. S.; Lee, S. A local unit root test in mean for financial time series. Journal of Statistical Computation and Simulation, 86, 4, 788-806, 2015.
-  Chaiboonsri, Chukiat; Wannapan, Satawat; Saosaovaphak, Anuphak. Economic and Business Cycle of India: Evidence from ICT Sector. p.29-43 in Tsounis, Nicholas; Vlachvei, Aspasia. Advances in Panel Data Analysis in Applied Economic Research. Cham, Switzerland, Springer Nature, 2018.
-  Cohen, E. Richard; Crowe, Kenneth M.; Dumond, Jesse W. M. The Fundamental Constants of Physics. NY, Interscience, 1957.
-  Cristofaro, Rodolfo de. The analytical solution to the problem of statistical induction. Statistica, 63, 2, 411-423, 2003.
-  D’Cunha, Juliet Gratia; Rao, Aruna K. Frequentist Comparison of the Bayesian Significance Test for Testing the Median of the Lognormal Distribution. InterStat, 2016, 02, 001, 1-25, 2016.
-  Diniz, Marcio; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael (2012). Cointegration: Bayesian Significance Test. Communications in Statistics - Theory and Methods, 41, 19, 3562-3574, 2012.
-  Diniz, Marcio; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael (2012). Unit Roots: Bayesian Significance Test. Communications in Statistics - Theory and Methods, 40, 23, 4200-4213, 2011.
-  Diniz, Marcio; Pereira, Carlos A. B.; Polpo, Adriano; Stern, Julio M.; Wechsler, Sergio. Relationship between Bayesian and Frequentist Significance Indices. International Journal for Uncertainty Quantification, 2, 2, 161-172, 2012.
-  Dubins Lester; Savage; Leonard Jimmie. How to Gamble If You Must. Inequalities for Stochastic Processes. NY, McGraw-Hill, 1965.
-  Esteves, Luis Gustavo; Izbicki, Rafael; Stern, Julio Michael; Stern, Rafael Bassi. The logical consistency of simultaneous agnostic hypothesis tests. Entropy, 18, 256, 2016
-  Esteves, Luis Gustavo; Izbicki, Rafael; Stern, Julio Michael; Stern, Rafael Bassi. Pragmatic Hypotheses in the Evolution of Science. Entropy, 21, 9, 883, 2019.
-  Evans, Michael (1997). Bayesian Inference Procedures Derived via the Concept of Relative Surprise. Communications in Statistics, 26, 1125-1143.
-  Fang, Shu Cheng; Rajasekera, Jay R.; Tsao, H.S.Jacob. Entropy Optimization and Mathematical Programming. Kluwer, Dordrecht, 1997.
-  Finetti, Bruno de. Theory of Probability. New York, Wiley.
-  Fossaluza, Victor; Lauretto, Marcelo de Souza; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. Combining Optimization and Randomization Approaches for the Design of Clinical Trials. Springer Proceedings in Mathematics and Statistics, 118, 173-184, 2015.
-  Fossaluza, Victor; Izbicki, Rafael; Silva, Gustavo Miranda da; Esteves, Luís Gustavo. Coherent Hypothesis Testing. The American Statistician, 71, 3, 242-248, 2017.
-  Fossaluza, Victor; Esteves, Luís Gustavo; Pereira, Carlos Alberto de Bragança. Estimating Multivariate Discrete Distributions Using Bernstein Copulas. Entropy, 20, 3, 194, 1-16, 2018.
-  Garcia, Manuel Valentim Pera; Humes, Carlos; Stern, Julio Michael. Generalized Line Criterion for Gauss-Seidel Method. Computational and Applied Mathematics, 22, 91-97, 2003.
-  García, Jesús E.; González-López, Verónica; Nelsen, Roger B. The Structure of the Class of Maximum Tsallis-Havrda-Chavat Entropy Copulas. Entropy, 18, 7, 264, 1-6, 2016.
-  Gelman, Andrew; Carlin, John B.; Stern, Hal S.; Rubin, Donald B. Bayesian Data Analysis. Chapman-Hall/ CRC, 2004.
-  Good, Irving John. Good Thinking. Univ. of Minnesota. 1983.
-  Hubert, Paulo; Lauretto, Marcelo de Souza; Stern, Julio Michael 2009. FBST for Generalized Poisson Distribution. AIP Conference Proceedings, 1193, 210-2019, 2009.
-  Hubert, Paulo; Padovese, Linilson; Stern, Julio Michael. A Sequential Algorithm for Signal Segmentation. Entropy, 20, 1, 55, 1-20.
-  Hubert, Paulo; Stern, Julio Michael. Probabilistic Equilibrium: A Review on the Application of MAXENT to Macroeconomic Models. Springer Proceedings in Mathematics and Statistics, 239, 187-197, 2018.
-  Hubert, Paulo; Killick, Rebecca; Chung, Alexandra; Padovese, Linilson R. A Bayesian binary algorithm for root mean squared-based acoustic signal segmentation. Journal of the Acoustical Society of America, 146, 3, 1799-1807, 2019.
-  Irony, Telba Z.; Lauretto, Marcelo de Souza; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. A Weibull Wearout Test: Full Bayesian Approach. p. 287-300 Y.Hayakawa, T.Irony, M.Xie, eds. Systems and Bayesian Reliability. Singapore, World Scientific, 2002.
-  Johnson, Ria; Chakrabarty, Dalia; O’Sullivan, Ewan; Raychaudhury, Somak. Comparing X-Ray and Dynamical Mass Profiles in the Early-Type Galaxy NGC 4636. The Astrophysical Journal, 706, 2, 706, 980-994, 2009.
-  Izbicki, Rafael; Fossaluza, Victor; Hounie, Ana Gabriela; Nakano, Eduardo Yoshio; Pereira, Carlos Alberto de Bragança. Testing allele homogeneity: the problem of nested hypotheses. BMC Genetics, 13, 103, 1-11, 2012.
-  Izbicki, Rafael; Esteves, Luis Gustavo. Logical consistency in simultaneous statistical test procedures. Logic Journal of the IGPL, 23, 732-758, 2015.
-  Jeffreys, Harold. Theory of Probability. Oxford, Clarendon Press. 1961, first ed. 1939.
-  Kadane, Joseph Born. Principles of Uncertainty. NY, Chapman-Hall/CRC, 2011.
-  Kadane, Joseph Born. Pragmatics of Uncertainty. NY, Chapman-Hall/CRC, 2016.
-  Kaplan, Stanley; Lin, James C. An Improved Condensation Procedure in Discrete Probability Distribution Calculations. Risk Analysis, 7, 15-19, 1987.
-  Kapur, Jagat Narain (1989). Maximum Entropy Models in Science and Engineering. New Delhi, John Wiley, 1989.
-  Kapur, Jagat Narain; Kesavan, Hiremagalur Krishnaswamy (1992). Entropy Optimization Principles with Applications. Boston, Academic Press, 1992.
-  Kaufmann, Arnold; Grouchko, Daniel; Cruon, R. Mathematical Models for the Study of the Reliability of Systems. NY, Academic Press, 1977.
-  Kelley, Carl Timothy. Iterative Methods for Linear and Nonlinear Equations. Philadelphia, SIAM, 1987.
-  Kelley, Carl Timothy. Iterative Methods for Optimization. Philadelphia, SIAM, 1987.
-  Kostrzewski, Maciej. On the Existence of Jumps in Financial Time Series. Acta Physica Polonica B, 43, 10, 2001-2019, 2012.
-  Lauretto, Marcelo de Souza; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael, Zacks, Shelemyahu. Full Bayesian Signicance Test Applied to Multivariate Normal Structure Models. Brazilian. Journal of Probability and Statistics, 17, 147-168, 2003.
-  Lauretto, Marcelo de Souza; Faria, Silvio; Pereira, Basilio de Bragança; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. The Problem of Separate Hypotheses via Mixtures Models. AIP Conference Proceedings, 954, 268-275, 2007.
-  Lauretto, Marcelo de Souza; Stern, Julio Michael. FBST for Mixture Model Selection. AIP Conference Proceedings, 803, 121-128, 2005.
Lauretto, Marcelo de Souza; Stern, Julio Michael. Testing Signicance in Bayesian Classifiers.
Frontiers in Artificial Intelligence and Applications, 132, 34-41, 2005.
-  Lauretto, Marcelo de Souza; Nakano, Fabio; Faria, Silvio; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. A Straightforward Multiallelic Signicance Test for the Hardy-Weinberg Equilibrium Law. Genetics and Molecular Biology, 32, 3, 619-625, 2009.
-  Lauretto, Marcelo de Souza; Nakano, Fabio; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. Intentional sampling by goal optimization with decoupling by stochastic perturbation. AIP Conference Proceedings, 1490, 189-201, 2014.
-  Lima, Adriano R.; Mello, Marcelo F.; Andreoli, Sérgio B.; Fossaluza, Victor; Araújo, Célia M. de; Jackowski, Andrea P.; Bressan, Rodrigo A.; Mari, Jair J. The Impact of Healthy Parenting As a Protective Factor for Posttraumatic Stress Disorder in Adulthood: A Case-Control Study. PLOS ONE, 9, 1, 1-9, e87117, 2014.
-  Loschi, Rosangela H.; Monteiro, João V. D; Rocha, Gustavo H. M. A.; Mayrink, Vinicius D. Testing and Estimating the Non-Disjunction Fraction in Meiosis I using Reference Priors. Biometrical Journal, 49, 6, 824-839, 2007.
-  Loschi, Rosangela H.; Santos, Cristiano C.; Arellano-Valle, Reinaldo B. Test procedures based on combination of Bayesian evidences for H0. Brazilian Journal of Probability and Statistics, 26, 4, 450-473, 2012.
-  Madruga, Maria Regina; Esteves, Luis Gustavo; Wechsler, Sergio. On the Bayesianity of Pereira-Stern Tests. Test, 10, 291-299, 2001.
-  Madruga, Maria Regina; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael (2003). Bayesian Evidence Test for Precise Hypotheses. Journal of Statistical Planning and Inference, 117, 185-198, 2003.
-  Mathis, Maria Alice de; Rosario, Maria Conceição do; Diniz, Juliana Belo; Torres, Albina Rodrigue; Shavitt, Roseli Gedanki; Ferrão, Ygor Arzeno; Fossaluza, Victor; Pereira, Carlos Alberto de Bragança; Miguel, Eurípedes Constantino. Obsessive-compulsive disorder: Influence of age at onset on comorbidity patterns. European Psychiatry, 23, 3, 187-194, 2008.
-  Montoya-Delgado, Luis E.; Irony, Telba Z.; Pereira, Carlos A. de B.; Whittle, Martin R. An Unconditional Exact Test for the Hardy-Weinberg Equilibrium Law: Sample-Space Ordering Using the Bayes Factor. Genetics, 158, 2, 875-883, 2001.
-  Maranhao, Viviane de Luca; Lauretto, Marcelo de Souza; Stern, Julio Michael (2012). FBST for Covariance Structures of Generalized Gompertz Models. AIP Conference Proceedings, 1490, 202-211, 2012.
-  Marcondes, Diego; Peixoto, Peixoto; Stern, Julio Michael. Assessing randomness in case assignment: The case study of the brazilian supreme court. Law, Probability and Risk, 18, 2-3, 97-114, 2019.
-  Minka, Thomas. Divergence measures and message passing. Technical report MSR-TR-2005-173, Microsoft Research Ltd., Cambridge, UK, 2005.
-  Nakano, Fabio; Pereira, Carlos Alberto de Bragança; Stern, Julio Michael; Whittle, Martin R. Genuine Bayesian Multiallelic Signicance Test for the Hardy-Weinberg Equilibrium Law. Genetics and Molecular Research, 4, 619-631, 2006.
-  Nelsen, Roger B. An Introduction to Copulas, 2nd ed. NY, Springer, 2006.
-  Oliveira, Natalia L.; Pereira, Carlos A. B.; Diniz, Marcio A; Polpo Adriano. A discussion on significance indices for contingency tables under small sample sizes. PLoS ONE, 13, 8, e0199102, 1-19, 2018.
-  Patriota, Alexandre Galvão. On Some Assumptions of the Null Hypothesis Statistical Testing. Educational and Psychological Measurement, 77, 3, 507-528, 2017.
-  Pawitan Yudi. In All Likelihood: Statistical Modelling and Inference Using Likelihood. Oxford University Press, 2001.
-  Pereira, Carlos Alberto de Bragança. Full Bayesian Significant Test (FBST). p.551-554 in: Lovric M. (eds) International Encyclopedia of Statistical Science, Berlin, Springer, 2011.
-  Pereira, Basilio de Bragança; Pereira, Carlos Alberto de Bragança. A Likelihood Aproach to Diagnostic Tests in Clinical Medicine. RevStat - Statistical Journal, 3, 1, 77-98, 2005.
-  Pereira, Basilio de Bragança; Pereira, Carlos Alberto de Bragança. Model Choice in Nonnested Families. Berlin, Springer. 2016.
-  Pereira, Carlos Alberto de Bragança, Barlow, Richard E. Medical diagnosis using influence diagrams. Networks, 20, 5, 565-577, 1990.
-  Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. Evidence and Credibility: Full Bayesian Significance Test for Precise Hypotheses. Entropy, 1, 99-110, 1999.
-  Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. A Dynamic Software Certification and Verification Procedure. ISAS-SCI’99 Proceedings, 2, 426-435, 1999.
-  Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. Model Selection and Regularization: Full Bayesian Approach. Environmetrics, 12, 6, 559-568, 2001.
-  Pereira, Carlos Alberto de Bragança; Stern, Julio Michael (2008); Wechsler, Segio. Can a Signicance Test be Genuinely Bayesian? Bayesian Analysis, 3, 79-100, 2008.
-  Pereira, Carlos Alberto de Bragança; Stern, Julio Michael. Special Characterizations of Standard Discrete Models. RevStat - Statistical Journal, 6, 199-230, 2008.
-  Pinto, Anna; Ventura, Laura. Approssimazioni Asintotiche di Ordine Elevato per Verifiche d’Ipotesi Bayesiani: Uno Studio per Dati di Sobrevvivenza. Università degli Studi di Padova, Dipartimento di Scienze Statistiche, 2012.
-  Ranzato, Giulia; Ventura, Laura. Biostatistica Bayesiana con “Matching Priors”. Università degli Studi di Padova, Dipartimento di Scienze Statistiche, 2018.
-  Rifo, Laura Leticia Ramos; Torres, Soledad. Full Bayesian Analysis for a Class of Jump-Diffusion Models. Communications in Statistics - Theory and Methods, 38, 8, 1262-1271, 2009.
-  Rifo, Laura Leticia Ramos; González-López, Veronica (2012). Full Bayesian Analysis for a Model of Tail Dependence. Communications in Statistics - Theory and Methods, 41, 22, 4107-4123, 2012.
-  Rincón, Sonia V. del; Rogers, Jeff; Widschwendter, Martin; Sun, Dahui; Sieburg, Hans B.; Spruck, Charles. Development and Validation of a Method for Profiling Post-Translational Modification Activities Using Protein Microarrays. PLoS ONE, 5, 6, e11332, 1-11, 2010.
-  Rodrigues, Josemar. Full Bayesian Significance Test for Zero-Inflated Distributions. Journal Communications in Statistics - Theory and Methods, 35, 2, 299-307, 2006.
-  Royall, Richard. Statistical Evidence: A Likelihood Paradigm. London, Chapman and Hall. 1997.
-  Ruli, Erlis; Sartori, Nicola; Ventura, Laura. Robust approximate Bayesian inference. Journal of Statistical Planning and Inference, 205, 10-22, 2020.
-  Saa, Olivia; Stern, Julio Michael. Auditable blockchain randomization tool. Proceedings, 33, 1, 17.1-17.6, 2019.
-  Santos, Natalia C. L.; Dias, Rosa; Alvesc, Diego; Melo, Brian Ganassin, Maria; Gomes, Luiz; Severid, Willia; Agostinho, Angelo. Trophic and limnological changes in highly fragmented rivers predict the decreasing abundance of detritivorous fish. Ecological Indicators, 110, 105933, 1-8, 2020.
-  Shackle, George Lennox Sharman. Uncertainty in Economics and Other Reflections. London, Cambridge Univ. Press, 1968.
-  Shackle, George Lennox Sharman. Decision, Order and Time in Human Affairs. London, Cambridge Univ. Press, 1969.
-  Shavitt, Roseli G.; Requena, Guaraci; Alonso, Pino; Zai, Gwyneth; Costa, Daniel L.C.; Pereira, Carlos Alberto de Bragança; Rosário, Maria Conceição; Morais, Ivanil; Fontenelle, Leonardo; Cappi, C.; Kennedy, James; Menchon, Jose M.; Miguel, Eurípides; Richter, Peggy M.A. Quantifying dimensional severity of obsessive-compulsive disorder for neurobiological research. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 79, 206-212, 2017.
-  Silva, Gustavo Miranda da; Esteves, Luis Gustav; Fossaluza, Victor; Izbicki, Rafael; Wechsler, Sergio. A Bayesian Decision-Theoretic Approach to Logically-Consistent Hypothesis Testing. Entropy, 17, 10, 6534-6559, 2015.
-  Silva, Ivair R. On the correspondence between frequentist and Bayesian tests. Communications in Statistics - Theory and Methods, 47, 14, 3477-3487, 2018.
-  Sikov, Anna; Stern, Julio M. Application of the full Bayesian significance test to model selection under informative sampling. Statistical Papers, 60, 89-104, 2019.
-  Spektor, Mikhail S.; Gluth, Sebastian; Fontanesi, Laura; Rieskamp, Jörg. How Similarity Between Choice Options Affects Decisions From Experience: The Accentuation-of-Differences Model. Psychological Review, 126, 1, 52-88, 2019.
-  Stern, Julio Michael. Simulated Annealing with a Temperature Dependent Penalty Function. ORSA Journal on Computing, 4, 311-319.
-  Stern, Julio Michael. Signicance Tests, Belief Calculi, and Burden of Proof in Legal and Scientic Discourse. Frontiers in Articial Intelligence and its Applications, 101, 139-147, 2003.
-  Stern, Julio Michael. Paraconsistent Sensitivity Analysis for Bayesian Signicance Tests. Lecture Notes in Artificial Intelligence, 3171, 134-143, 2004.
-  Stern, Julio Michael. Cognitive Constructivism, Eigen-Solutions, and Sharp Statistical Hypotheses. Cybernetics & Human Knowing, 14, 1, 9-36, 2007.
-  Stern, Julio Michael. Language and the Self-Reference Paradox. Cybernetics & Human Knowing, 14, 4, 71-92, 2007.
-  Stern, Julio Michael. Decoupling, Sparsity, Randomization, and Objective Bayesian Inference. Cybernetics & Human Knowing, 15, 2, 49-68, 2008.
-  Stern, Julio Michael. Cognitive Constructivism and the Epistemic Significance of Sharp Statistical Hypotheses in Natural Sciences. arXiv:1006.5471. Tutorial text for MaxEnt 2008 - The 28th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Boracéia, São Paulo, Brazil, July 6-11, 2008.
-  Stern, Julio Michael. Symmetry, Invariance and Ontology in Physics and Statistics. Symmetry, 3, 3, 611-635, 2011.
-  Stern, Julio Michael. Constructive Verification, Empirical Induction, and Falibilist Deduction: A Threefold Contrast. Information, 2, 635-650, 2011.
-  Stern, Julio Michael. Jacob’s Ladder and Scientific Ontologies. Cybernetics & Human Knowing, 21, 3, 9-43, 2014.
-  Stern, Julio Michael. Cognitive-Constructivism, Quine, Dogmas of Empiricism, and Münchhausen’s Trilemma. In: Adriano Polpo, Francisco Louzada, Laura L. R. Rifo, Julio M. Stern, and Marcelo Lauretto (eds.). Interdisciplinary Bayesian Statistics: EBEB 2014, Springer Proceedings in Mathematics and Statistics, 118, 55-68, 2015.
-  Stern, Julio Michael. Continuous versions of Haack’s Puzzles: Equilibria, Eigen-States and Ontologies. Logic Journal of the IGPL, 25, 4, 604-631, 2017.
-  Stern, Julio Michael. Jacob’s Ladder: Logics of Magic, Metaphor and Metaphysics: Narratives of the Unconscious, the Self, and the Assembly. Sophia, published Online First June 7, 2017. DOI: 10.1007/s11841-017-0592-y.
-  Stern, Julio Michael. Verstehen (causal/interpretative understanding), Erklären (law-governed description/prediction), and Empirical Legal Studies. Journal of Institutional and Theoretical Economics, 174, 105-114, 2018.
-  Stern, Julio Michael. Karl Pearson on Causes and Inverse Probabilities: Renouncing the Bride, Inverted Spinozism and Goodness-of-Fit. South American Journal of Logic, 4, 1, 219-252, 2018.
-  Stern, Julio Michael; Izbicki, Rafael; Esteves, Luis Gustavo; Stern, Rafael Bassi. Logically-Consistent Hypothesis Testing and the Hexagon of Oppositions. Logic Journal of the IGPL, 25, 741-757, 2018.
-  Stern, Julio Michael; Colla, Ernesto Coutinho. Factorization of Bayesian Networks. p.275-294 in Nakamatsu, K.; Phillips-Wren, G.; Jain, L.C.; Howlett, R.J. eds. New Advances in Intelligent Decision Technologies. Heidelberg, Springer, 2009.
-  Stern, Julio Michael; Pereira, Carlos Alberto de Bragança (2014). Bayesian Epistemic Values: Focus on Surprise, Measure Probability! Logic Journal of the IGPL, 22, 236-254, 2014.
-  Stern, Julio Michael; Vavasis, Stephen Andrew. Active Set Methods for Problems in Column Block Angular Form. Computational and Applied Mathematics, 12, 199-226, 1994.
-  Stern, Julio Michael; Zacks, Shelemyahu. Testing the Independence of Poisson Variates under the Holgate Bivariate Distribution: The Power of a New Evidence Test. Statistical and Probability Letters, 60, 313-320, 2002.
-  Thulin, Mans. Decision-theoretic justifications for Bayesian hypothesis testing using credible sets. Journal of Statistical Planning and Inference, 146, 133-138, 2014.
-  Ventura, Laura; Ruli, Erlis; Racugno, Walter. A note on approximate Bayesian credible sets based on modified loglikelihood ratios. Statistics and Probability Letters, 83, 11, 2467-2472, 2013.
-  Ventura, Laura; Reid, Nancy. Approximate Bayesian computation with modified log-likelihood ratios. METRON, 72, 231-245, 2014.
-  Ventura, Laura; Racugno, Walter. Pseudo-Likelihoods for Bayesian Inference. p.205-220 in DiBattista T., Moreno E., Racugno W. (eds). Topics on Methodological and Applied Statistical Inference. Berlin, Springer, 2016.
-  Vieland, V. J.; Chang, H. No evidence amalgamation without evidence measurement. Synthese. 196, 3139-3161, 2019.
-  Vikas, K.; Rao, Aruna K. Full Bayesian Empirical Likelihood Significance Test for Equality of Medians. InterStat, 2016, 01, 001, 1-9, 2016.
-  Vosseler, Alexander; Weber, Enzo. (2016). Bayesian analysis of periodic unit roots in the presence of a break. Applied Economics, 49, 38, 3841-3862, 2016.
-  Wechsler, Sergio; Pereira, Carlos Alberto de Bragança; Marques, Paulo C. Birnbaum’s Theorem Redux. AIP Conference Proceedings, 1073, 96-100, 2008.
-  Wittenburg, Dörte; Teuscher, Friedrich; Klosa, Jan; Reinsch, Norbert. Covariance Between Genotypic Effects and its Use for Genomic Inference in Half-Sib Families. G3 - Genes Genomes Genetics, 6, 9, 2761-2772, 2016.
-  Zellner, Arnold. Introduction to Bayesian Inference in Econometrics. NY, Wiley, 1971.