Archive
Adaptively selecting an empirical Bayes reference class
F. A. Aghababazadeh, M. Alvo, and D. R. Bickel, “Estimating the local false discovery rate via a bootstrap solution to the reference class problem,” Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/34295 (2016). 2016 preprint
Empirical Bayes software (R packages)
Empirical Bayes software packages:
Frequentist inference principles
On some principles of statistical inference.
Int. Stat. Rev. 83 (2015), no. 2, 293–308.
62A01 (62F05 62F15 62F25)
While agreeing with other frequentists on the necessity of guaranteeing good performance over repeated sampling, Reid and Cox also value neglected rules of inference such as the conditionality principle. Against the steady advance of nonparametric methods, Reid and Cox point to the interpretive power of parametric models.Frequentist decision theory is only mentioned. Glimpses of the authors’ perspectives on that appear in [D. R. Cox, Principles of statistical inference, Cambridge Univ. Press, Cambridge, 2006 (8.2); MR2278763 (2007g:62007)] and [N. M. Reid, Statist. Sci. 9 (1994), no. 3, 439–455; MR1325436 (95m:01020)].On the Bayes front, Reid and Cox highlight the success frequentist methods have enjoyed in scientific applications as a decisive victory over those Bayesian methods that are most consistent with their subjectivist foundations. Indeed, no one can deny what C. Howson and P. Urbach call the “social success” of frequentist methods [Scientific reasoning: the Bayesian approach, third edition, Open Court, Chicago, IL, 2005 (p. 9)]. Reid and Cox do not attribute their widespread use in scientific practice to political factors.
Rather, for scientific inference as opposed to individual decision making, they find frequentist methods more suitable in principle than fully Bayesian methods. For while the need for an agent to reach a decision recognizes no line between models of the phenomena under study and models of an agent’s thought, science requires clear reporting on the basis of the former without introducing biases from the latter. Although subjective considerations admittedly come into play in interpreting reports of statistical analyses, a dependence of the reports themselves on such considerations conflicts with scientific methodology. In short, the Bayesian theories supporting personal inference are irrelevant as far as science is concerned even if they are useful in personal decision making. This viewpoint stops short of that of Philip Stark, who went as far as to call the practicality of that private application of Bayesian inference into question [SIAM/ASA J. Uncertain. Quantif. 3 (2015), no. 1, 586–598; MR3372107].
On reference priors designed to minimize subjective input, Reid and Cox point out that those that perform well with low-dimensional parameters can fail in high dimensions. Eliminating the prior entirely leads to the pure likelihood approach, which, based on the strong likelihood principle, limits the scope even further, to problems with a scalar parameter of interest and no nuisance parameters [A. W. F. Edwards, Likelihood. An account of the statistical concept of likelihood and its application to scientific inference, Cambridge Univ. Press, London, 1972; MR0348869 (50 #1363)]. More recent developments of that approach were explained by R. M. Royall [Statistical evidence, Monogr. Statist. Appl. Probab., 71, Chapman & Hall, London, 1997; MR1629481 (99f:62012)] and C. A. Rohde [Introductory statistical inference with the likelihood function, Springer, Cham, 2014 (Chapter 18); MR3243684].
Reid and Cox see some utility in Bayesian methods that have good performance by frequentist standards, noting that such performance can require the prior to depend on which parameter happens to be of interest and, through model checking, on the data. Such dependence raises the question, “Is this, then, Bayesian? The prior distribution will then not represent prior knowledge of the parameter in [that] case, but an understanding of the model” [T. Schweder and N. L. Hjort, Scand. J. Statist. 29 (2002), no. 2, 309–332; MR1909788 (2003d:62085)].
Reviewed by David R. Bickel
This review first appeared at “On some principles of statistical inference” (Mathematical Reviews) and is used with permission from the American Mathematical Society.
Coherent inference after checking a prior
D. R. Bickel, “Bayesian revision of a prior given prior-data conflict, expert opinion, or a similar insight: A large-deviation approach,” Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/34089/ (2015). 2015 preprint
Meaningful constraints and meaningless priors
Constraints versus priors.
SIAM/ASA J. Uncertain. Quantif. 3 (2015), no. 1, 586–598.
62A01 (62C10 62C20 62G15)
In this lucid expository paper, Stark advances several arguments for using frequentist methods instead of Bayesian methods in statistical inference and decision problems. The main examples involve restricted-parameter problems, those of inferring the value of a parameter of interest that is constrained to lie in an unusually restrictive set. When the parameter is restricted, frequentist methods can lead to solutions markedly different from those of Bayesian methods. For even when the prior distribution is a default intended to be weakly informative, it actually carries substantial information.
Stark calls routine Bayesian practice into question since priors are not selected according to the analyst’s beliefs but rather for reasons that have no apparent support from the Dutch book argument, the featured rationale for Bayesianism as a rational norm (pp. 589–590; [see D. V. Lindley, Understanding uncertainty, revised edition, Wiley Ser. Probab. Stat., Wiley, Hoboken, NJ, 2014; MR3236718]). Uses of the prior beyond the scope of the paper include those encoding (1) empirical Bayes estimates of parameter variability [e.g., B. Efron, Large-scale inference, Inst. Math. Stat. Monogr., 1, Cambridge Univ. Press, Cambridge, 2010; MR2724758 (2012a:62006)], (2) the beliefs of subject-matter experts [e.g., A. O’Hagan et al., Uncertain judgements: eliciting experts’ probabilities, Wiley, West Sussex, 2006, doi:10.1002/0470033312], or (3) the beliefs of archetypical agents of wide scientific interest [e.g., D. J. Spiegelhalter, K. R. Abrams and J. P. Myles, Bayesian approaches to clinical trials and health-care evaluation, Wiley, West Sussex, 2004 (Section 5.5), doi:10.1002/0470092602].
Stark finds Bayesianism to lack not only normative force but also descriptive power. He stresses that he does not know anyone who updates personal beliefs according to Bayes’s theorem in everyday life (pp. 588, 590).
In the conclusions section, Stark asks, “Which is the more interesting question: what would happen if Nature generated a new value of the parameter and the data happened to remain the same, or what would happen for the same value of the parameter if the measurement were repeated?” For the Bayesian who sees parameter distributions more in terms of beliefs than random events, the missing question is, “What should one believe about the value of a parameter given what happened and the information encoded in the prior and other model specifications?” That question would interest Stark only to the extent that the prior encodes meaningful information (p. 589).
Reviewed by David R. Bickel
This review first appeared at “Constraints versus priors” (Mathematical Reviews) and is used with permission from the American Mathematical Society.
Understanding Uncertainty (by Lindley)—a review
Lindley, Dennis V.
Understanding uncertainty.
Revised edition. Wiley Series in Probability and Statistics. John Wiley & Sons, Inc., Hoboken, NJ, 2014. xvi+393 pp. ISBN: 978-1-118-65012-7
62A99 (62C05 62C10)
In Understanding uncertainty, Dennis Lindley ably defends subjective Bayesianism, the thesis that decisions in the presence of uncertainty can only be guaranteed to cohere if made according to probabilities as degrees of someone’s beliefs. True to form, he excludes all other mathematical theories of modeling uncertainty, including subjective theories of imprecise probability that share the goal of coherent decision making [see M. C. M. Troffaes and G. de Cooman, Lower previsions, Wiley Ser. Probab. Stat., Wiley, Chichester, 2014; MR3222242].
In order to engage everyone interested in making better decisions in the presence of uncertainty, Lindley writes without the citations and cluttered notation of a research paper. His straightforward, disarming style advances the thesis that subjective probability saves uncertainty from getting lost in the fog of reasoning in natural-language arguments. A particularly convincing argument is that the reader who makes decisions in conflict with the strict Bayesian viewpoint will be vulnerable to a Dutch book comprising undesirable consequences regardless of the true state of the world (5.7). The axioms needed for the underlying theorem are confidently presented as self-evident.
Like many strict Bayesians, Lindley makes no appeal to epistemological or psychological literature supporting the alignment of belief and probability. In fact, he dismisses studies indicating that actual human beliefs can deviate markedly from the requirements of strict Bayesianism, likening them to studies indicating that people make errors in arithmetic (2.5; 9.12).
The relentlessly pursued thesis is nuanced by the clarification that strict Bayesianism is not an inviolable recipe for automatic decisions but rather a box of tools that can only be used effectively when controlled by human judgment or “art” in modeling (11.7). For example, when Lindley intuitively finds that the prior distribution under his model conflicts with observations, he reframes its prior probabilities as conditional on the truth of the original model by crafting a larger model. Such ingenuity demonstrates that Bayesian probability calculations cannot shackle his actual beliefs. (This suggests that mechanically following the Dutch book argument to the point of absurdity might not discredit strict Bayesianism as decisively as thought.) Similarly, Frank Lad, called “the purest of the pure” [G. Shafer, J. Am. Stat. Assoc. 94 (1999), no. 446, 645–656 (pp. 648–649), doi:10.1080/01621459.1999.10474158] and the best-informed [D. V. Lindley, J. Royal Stat. Soc. Ser. D 49 (2000), no. 3, 293–337] of the advocates of this school, permits replacing a poorly predicting model with one that reflects “a new understanding”, an enlightenment that no algorithm can impart [F. Lad, Operational subjective statistical methods, Wiley Ser. Probab. Statist. Appl. Probab. Statist., Wiley, New York, 1996 (6.6.4); MR1421323 (98m:62009)]. Leonard Savage, a leading critic of non-Bayesian statistical methods, likewise admitted that he was “unable to formulate criteria for selecting these small worlds [in which strict Bayesianism applies] and indeed believe[d] that their selection may be a matter of judgment and experience about which it is impossible to enunciate complete and sharply defined general principles” [L. J. Savage, The foundations of statistics, Wiley, New York, 1954 (2.5); MR0063582 (16,147a)]. The Bayesian lumberjacks have evidently learned when to stop chopping and sharpen the axe. This recalls the importance of the skill of the scientist as handed down and developed within the guild of scientists and never quite articulated, let alone formalized [M. Polanyi, Personal knowledge: towards a post-critical philosophy, Univ. Chicago Press, Chicago, IL, 1962]. The explicit acknowledgement of the role of this tacit knowledge in science may serve as a warning against relying on statistical models as if they were not only useful but also right [see M. van der Laan, Amstat News 2015, no. 452, 29–30].
While the overall argument for strict Bayesianism will command the assent of many readers, some will wonder whether there are equally compelling counter-arguments that would explain why so few statisticians work under that viewpoint. That doubt will be largely offset by the considerable authority Lindley has earned as one of the preeminent developers of the statistics discipline as it is known today. His many enduring contributions to the field include two that shed light on the chasm between Bayesian and frequentist probabilities: (1) the presentation of what is known as “Lindley’s paradox” [D. V. Lindley, Biometrika 44 (1957), no. 1-2, 187–192, doi:10.1093/biomet/44.1-2.187] and (2) specifying the conditions a scalar-parameter fiducial or confidence distribution must satisfy to be a Bayesian posterior distribution [D. V. Lindley, J. Royal Stat. Soc. Ser. B 20 (1958), 102–107; MR0095550 (20 #2052)].
Treading into unresolved controversies well outside his discipline, Lindley shares his simple philosophy of science and offers his opinions on how to apply Bayesianism to law, politics, and religion. He invites his readers to share his hope that if people communicate their beliefs and interests in strict Bayesian terms, they would quarrel less (1.7; 10.7), especially if they adopt his additional advice to consider their own religious beliefs to be uncertain (1.4). Lindley even holds forth the teaching that Jesus is the Son of God as having a probability equal to each reader’s degree of belief in its truth but stops short of assessing the utilities needed to place Pascal’s Wager (1.2).
Graduate students in statistics will benefit from Lindley’s introductions to his paradox, explained in Section 14.4 to discredit frequentist hypothesis testing, and the conglomerable rule in Section 12.9. These friendly and concise introductions could effectively supplement a textbook such as [J. B. Kadane, Principles of uncertainty, Texts Statist. Sci. Ser., CRC Press, Boca Raton, FL, 2011; MR2799022 (2012g:62001)], a much more detailed appeal for strict Bayesianism.
On the other hand, simpler works such as [J. S. Hammond, R. L. Keeney and H. Raiffa, Smart choices: a practical guide to making better decisions, Harvard Bus. School Press, Boston, MA, 1999] may better serve as stand-alone guides to mundane decision making. Bridging the logical gap between decision making rules of thumb and mathematical statistics, Understanding uncertainty excels as a straightforward and sensible defense of the strict Bayesian viewpoint. Appreciating Lindley’s stance in all its theoretical simplicity and pragmatic pliability is essential for grasping both the recent history of statistics and the more complex versions of Bayesianism now used by statisticians, scientists, philosophers, and economists.
{For the original edition see [D. V. Lindley, Understanding uncertainty, Wiley, Hoboken, NJ, 2006].}
Reviewed by David R. Bickel
Inference after checking the prior & sampling model
D. R. Bickel, “Inference after checking multiple Bayesian models for data conflict and applications to mitigating the influence of rejected priors,” International Journal of Approximate Reasoning 66, 53–72 (2015). Simple explanation | Published version | 2014 preprint | Slides
The proposed procedure combines Bayesian model checking with robust Bayes acts to guide inference whether or not the model is found to be inadequate:
- The first stage of the procedure checks each model within a large class of models to determine which models are in conflict with the data and which are adequate for purposes of data analysis.
- The second stage of the procedure applies distribution combination or decision rules developed for imprecise probability.
This proposed procedure is illustrated by the application of a class of hierarchical models to a simple data set.
The link Simple explanation was added on 6 June 2017.
Fiducial nonparametrics
Sonderegger, Derek L.; Hannig, Jan
Fiducial theory for free-knot splines. Contemporary developments in statistical theory, 155–189,
Springer Proc. Math. Stat., 68, Springer, Cham, 2014.
62F12 (62F10 62F99 65D07)
The research reported reflects the recent surge in developments of Fisher’s fiducial argument [S. Nadarajah, S. Bityukov and N. Krasnikov, Stat. Methodol. 22 (2015), 23–46; MR3261595]. The work of this chapter is carried out within the framework of generalized fiducial inference [J. Hannig, Statist. Sinica 19 (2009), no. 2, 491–544; MR2514173 (2010h:62071)], which is built on the functional-model formulation of fiducial statistics [A. P. Dawid, M. Stone and M. Stone, Ann. Statist. 10 (1982), no. 4, 1054–1074; MR0673643 (83m:62008)] rather than on the broadly equivalent confidence-based tradition beginning with [G. N. Wilkinson, J. Roy. Statist. Soc. Ser. B 39 (1977), no. 2, 119–171; MR0652326 (58 #31491)] and generalized by [E. E. M. van Berkum, H. N. Linssen and D. Overdijk, J. Statist. Plann. Inference 49 (1996), no. 3, 305–317; MR1381161 (97k:62007)].
{For the entire collection see MR3149911.}
Reviewed by David R. Bickel
This review first appeared at “Fiducial theory for free-knot splines” (Mathematical Reviews) and is used with permission from the American Mathematical Society.
Maximum entropy over a set of posteriors
D. R. Bickel, “Blending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing,” Statistical Methods & Applications 24, 523-546 (2015). Published article | 2012 preprint | 2011 preprint | Slides | Simple explanation
This framework of statistical inference facilitates the development of new methodology to bridge the gap between the frequentist and Bayesian theories. As an example, a simple and practical method for combining p-values with a set of possible posterior probabilities is provided.
In this general approach, Bayesian inference is used when the prior distribution is known, frequentist inference is used when nothing is known about the prior, and both types of inference are blended according to game theory when the prior is known to be a member of some set. (The robust Bayes framework represents knowledge about a prior in terms of a set of possible priors.) If the benchmark posterior that corresponds to frequentist inference lies within the set of Bayesian posteriors derived from the set of priors, then the benchmark posterior is used for inference. Otherwise, the posterior within that set that minimizes the cross entropy to the benchmark posterior is used for inference.
David Bickel, statistician 
You must be logged in to post a comment.