About the Statomics Lab

2 April 2017 Comments off

The Complexity and Statistics Research Lab was called “The Statomics Lab” until 4 February 2017. The word statomics abbreviates statistical inference and computation in genomics. David Bickel launched the lab in June of 2007 at the Ottawa Institute of Systems Biology.

Categories: Fragments

Entropies of a posterior of the success probability

1 February 2017 Comments off

Kelbert, M.; Mozgunov, P.
Asymptotic behaviour of the weighted Renyi, Tsallis and Fisher entropies in a Bayesian problem. (English summary)
Eurasian Math. J. 6 (2015), no. 2, 6–17.
94A17 (62B10 62C10)

This paper considers a weighted version of the differential entropy of the posterior distribution of the probability of success conditional on the observed value of a binomial random variable. The uniform (0,1)prior distribution of the success probability is used to derive large-sample results.
The weighting function allows emphasizing some values of the parameter more than other values. For example, since the success probability value of 1/2 has special importance in many applications, that parameter value may be assigned a higher weight than the others. This differs from the more common Bayesian approach of assigning more prior probability to certain parameter values.
The author proves asymptotic properties not only of the weighted differential entropy but also of weighted differential versions of the Renyi, Tsallis, and Fisher definitions of entropy or information. The results are concrete in that they are specifically derived for the posterior distribution of the success probability given the uniform prior.

Reviewed by David R. Bickel

This review first appeared at “Asymptotic behaviour of the weighted Renyi, Tsallis and Fisher entropies in a Bayesian problem” (Mathematical Reviews) and is used with permission from the American Mathematical Society.
Categories: reviews

Should simpler distributions have more prior probability?

7 January 2017 Comments off

D. R. Bickel, “Computable priors sharpened into Occam’s razors,” Working Paper, University of Ottawa, <hal-01423673> https://hal.archives-ouvertes.fr/hal-01423673 (2016). 2016 preprint

Categories: complexity, preprints

Inference after eliminating Bayesian models of insufficient evidence

1 December 2016 Comments off

“Inference under the entropy-maximizing Bayesian model of sufficient evidence”

The Third International Conference on Mathematical and Computational Medicine

Columbus, Ohio

David R. Bickel

18 May 2016

Entropy sightings

1 November 2016 Comments off
Varadhan, Srinivasa R. S.
Entropy and its many avatars. (English summary)
J. Math. Soc. Japan 67 (2015), no. 4, 1845–1857.
94A17 (37A35 60-02 60K35 82B05)

The author, a chief architect of the theory of large deviations, chronicles several manifestations of entropy. It made appearances in the realms indicated by these section headings:

  • Entropy and information theory
  • Entropy and dynamical systems
  • Relative entropy and large deviations
  • Entropy and duality
  • Log Sobolev inequality
  • Gibbs states
  • Interacting particle systems

The topics are connected whenever a concept introduced in one section is treated in more depth in a later section. In this way, relative entropy is seen to play a key role in large deviations, Gibbs states, and systems of interacting particles.
Less explicit connections are left to the reader’s enjoyment and education. For example, the relation between Boltzmann entropy and Shannon entropy in the information theory section is a special case both of Sanov’s theorem, presented in the section on large deviations, and of the relation of free energy and relative entropy, in the section on Gibbs states.
The paper ends with a tribute to Professor Kiyosi Itô.

Reviewed by David R. Bickel


  1. J. Axzel and Z. Daroczy, On Measures of Information and Their Characterizations, Academic Press, New York, 1975. MR0689178 
  2. L. Boltzmann, Über die Mechanische Bedeutung des Zweiten Hauptsatzes der Wärmetheorie, Wiener Berichte, 53 (1866), 195–220.
  3. R. Clausius, Théorie mécanique de la chaleur, lère partie, Paris: Lacroix, 1868.
  4. H. Cramer, On a new limit theorem in the theory of probability, Colloquium on the Theory of Probability, Hermann, Paris, 1937.
  5. J. D. Deuschel and D. W. Stroock, Large deviations, Pure and Appl. Math., 137, Academic Press, Inc., Boston, MA, 1989, xiv+307 pp.  MR0997938 
  6. M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, IV, Comm. Pure Appl. Math., 36 (1983), 183–212.  MR0690656 
  7. A. Feinstein, A new basic theorem of information theory, IRE Trans. Information Theory PGIT-4 (1954), 2–22.  MR0088413 
  8. L. Gross, Logarithmic Sobolev inequalities, Amer. J. Math.,  97 (1975), 1061–1083.  MR0420249 
  9. M. Z. Guo, G. C. Papanicolaou and S. R. S. Varadhan, Nonlinear diffusion limit for a system with nearest neighbor interactions, Comm. Math. Phys., 118 (1988), 31–59.  MR0954674 
  10. A. I. Khinchin, On the fundamental theorems of information theory, Translated by Morris D. Friedman, 572 California St., Newtonville MA 02460, 1956, 84 pp.  MR0082924 
  11. A. N. Kolmogorov, A new metric invariant of transitive dynamical systems and automorphisms of Lebesgue spaces, (Russian) Topology, ordinary differential equations, dynamical systems, Trudy Mat. Inst., Steklov., 169 (1985), 94–98, 254.  MR0836570 
  12. O. Lanford, Entropy and equilibrium states in classical statistical mechanics, Statistical Mechanics and Mathematical Problems, Lecture notes in Physics, 20, Springer-Verlag, Berlin and New York, 1971, 1–113.
  13. D. S. Ornstein, Ergodic theory, randomness, and dynamical systems, James K. Whittemore Lectures in Mathematics given at Yale University, Yale Mathematical Monographs, No. 5. Yale University Press, New Haven, Conn.-London, 1974, vii+141 pp. MR0447525 
  14. I. N. Sanov, On the probability of large deviations of random magnitudes, (Russian) Mat. Sb. (N. S.), 42 (84) (1957), 11–44. MR0088087 
  15. C. E. Shannon, A mathematical theory of communication, Bell System Tech. J., 27 (1948), 379–423, 623–656.  MR0026286 
  16. Y. G. Sinai, On a weak isomorphism of transformations with invariant measure, (Russian) Mat. Sb. (N.S.), 63 (105) (1964), 23–42.  MR0161961 
  17. H. T. Yau, Relative entropy and hydrodynamics of Ginzburg-Landau models, Lett. Math. Phys., 22 (1991), 63–80.  MR1121850 
This list reflects references listed in the original paper as accurately as possible with no attempt to correct error.
This review first appeared at “Entropy and its many avatars” (Mathematical Reviews) and is used with permission from the American Mathematical Society.
Categories: reviews

A Bayesian approach to informing decision makers

23 September 2016 Comments off

Undergraduate research project or internship

2 September 2016 Comments off

Acquire a statistical bioinformatics skill set by developing novel scientific software in the frontiers of genomics for high impact on medical science. Learn to analyze genomics data with newly created statistical methods. Make new biostatistics software accessible worldwide by improving the usability and functionality of the Statomics Lab’s data analysis code and by adding documentation. Providing scientists with these reliable biostatistical tools can advance medical research by improving the accuracy of conclusions drawn from genomics and clinical data.

Scientific breakthroughs from genome-sequencing projects brought the realization that reliable interpretation of the resulting information makes unprecedented demands for contemporaneous advances in computation and mathematical modeling. As the complexity of genomic data sets drives innovative statistics research, the Statomics Lab (http://davidbickel.com) aims to develop and apply novel methodology and algorithms to solve current problems in analyzing gene-expression, proteomics, metabolomics, SNP, ChIP-chip, and/or clinical data.

Intellectual curiosity and high mathematical aptitude are essential, as is the ability to quickly code and debug computer programs. Strong self motivation and good communication skills are also absolutely necessary. The following qualities are desirable but not required: coursework in bioinformatics, computer science, numerical methods, numerical analysis, software engineering, statistics, and/or biology; familiarly with BUGS, R, S-PLUS, C, Fortran, and/or LaTeX; experience with UNIX or Linux.

To be considered, send a PDF CV that has your GPA and contact information of two references to dbickel@uOttawa.ca with either “research project” or “internship” in the Subject line of the message and with a cover letter in the body of the message. Only those students selected for further consideration will receive a response.

Categories: applications welcome