SYST 664 / OR 664 / CSI 674

Bayesian Inference and Decision Theory

Kathryn Blackmond Laskey
Department of Systems Engineering and Operations Research
George Mason University

Spring, 2023
ENT 276 and online
Monday, 4:30-7:10 PM


Bayesian decision theory provides a unified and intuitively appealing approach to drawing inferences from observations and making rational, informed decisions.   Bayesians view statistical inference as a problem in belief dynamics, of using evidence about a phenomenon to revise and update knowledge about it. Bayesian statistics is a scientifically justifiable way to integrate informed expert judgment with empirical data.  For a Bayesian, statistical inference cannot be treated entirely independently of the context of the decisions that will be made on the basis of the inferences.  In recent years, Bayesian methods have become increasingly common in a variety of disciplines that rely heavily on data. This course introduces students to Bayesian theory and methodology, including modern computational methods for Bayesian inference.  Students will learn the commonalities and differences between the Bayesian and frequentist approaches to statistical inference, how to approach a statistics problem from the Bayesian perspective, and how to combine data with informed expert judgment in a sound way to derive useful and policy-relevant conclusions.  Students will learn the necessary theory to develop a firm understanding of when and how to apply Bayesian and frequentist methods, and will also learn practical procedures for developing statistical models for phenomena, drawing inferences, and evaluating evidential support for hypotheses.  The course covers fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, Markov Chain Monte Carlo methods for approximating the posterior distribution, Bayesian hierarchical models, and other key topics.  Graphical models are introduced for representing complex probability and decision problems by specifying them in modular components.  Assignments make use of modern computational techniques and focus on applying the methods to practical problems.

Syllabus

Delivery Mode

The class will be taught in person and synchronously on line Mondays at 4:30 PM. All classes will be recorded so you can listen and review at your convenience.

Lecture Notes

Lecture notes for each unit will be made available before the first class of the unit.  This class was last offered in Spring 2022.  Links to the previous notes are provided for those who like to read ahead. There may be be changes both in the order of topics covered and the notes themselves.

Homework Assignments

Homework is due 11:59PM on the assigned due date. If it is submitted before 23:59 the day after the due date, you will receive 75% credit.  If it is submitted up to 1 week late, you will receive 50% credit.  If you have extenuating circumstances, please contact me in advance, and I will consider giving you additional time to complete the assignment for partial credit. Assignments will be posted here and on Blackboard. Please submit your assignments through Gradescope.

Exams

Study Aids

Sites of Possible Interest

A recent web search on "Bayesian" yielded 1.9 billion hits (up from 39 million in 2019, 12 million in 2017, and 5 million in 2009).  When you have some time, it would be a good idea to throw in a modifier or two to reflect your individual interests and browse.  There is a wealth of interesting and useful material.  I've culled a few tidbits to get you started.  Many of these sites also contain useful links to other Bayesian sites.
Professional Societies:
International Society for Bayesian Analysis
American Statistical Association Section on Bayesian Statistical Science
A Non-Comprehensive Sampling of Statistics Departments with Bayesian Orientation:
Carnegie Mellon University Department of Statistics and Data Science
Duke University Department of Statistical Science
University of Minnesota School of Statistics
University of Washington Department of Statistics
A Non-Comprehensive List of Free Bayesian Statistical Software:
MCMCpack (an R package for Bayesian analysis)
Bayesian Analysis Using Gibbs Sampling (BUGS)
Just Another Gibbs Sampler (JAGS)
Stan platform for statistical modeling and high-performance statistical computation
Github site with software and data to replicate examples in Peter Hoff text
 Companion software to Peter Lee text
UnBBayes open source plugin framework for  Probabilistic Graphical Models
Miscellaneous
Andrew Gelma's Blog
The Bayesian Songbook (includes Frequentist Frenzy by world renowned songwriter Kathryn Laskey)
Bayesians Worldwide (list of self-identified Bayesians maintained by ISBA)