OR 664 / SYST 664 / CSI 674

Bayesian Inference and Decision Theory

Spring Semester, 2022
IN 132 and online
Monday 4:30-7:10 PM
 
The objective of this course is to introduce students to Bayesian inference and decision making and to provide practical experience in applications from information technology and engineering.  Students will learn the fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, and methods for approximating the posterior distribution.  Graphical models are introduced for representing complex probability and decision models by specifying modular components.  An overview is given of expected utility and decision theory.  Students apply the methods to a variety of practical problems.  Modern Bayesian computational methods are introduced.

Instructor:  Kathryn Blackmond Laskey
email:  klaskey@gmu.edu
web:  http://www.seor.gmu.edu/~klaskey
phone:  703-993-1644
Office hours:  Mondays 3:00 to 4:00 PM (online or in person), Wednesdays 4:00 to 5:00 PM (online only) or by appointment
Office location: ENGR 2214
Prerequisites:
OR 542 or STAT 544 or STAT 554 or equivalent (a strong grounding in probability with calculus; skill in elementary data analysis; basic programming skills)

Textbook

I teach from my notes, not from a textbook. The material I cover follows the Hoff text fairly closely, but the Hoff text is not as readable as I would like. Most students like to have a text as a supplement to my notes. The first two listed texts have electronic versions available from the library.

Primary text (recommended): This book, published in 2009, provides about the right level of coverage and is a reasonably up-to-date treatment.  An electronic edition of this book is available to George Mason University students and faculty from the university library. Computer code is available at the link below for most of the examples in the book.

Hoff, Peter D., A First Course in Bayesian Statistical Methods. Springer, 2009.

Secondary text (recommended):  This recently published book was written primarily for social scientists.  It is accessible, well-written, and gives a comprehensive treatment beginning from the very basics through sophisticated hierarchical Bayesian models. An electronic edition of this book is available to George Mason University students and faculty from the university library. Computer code is available at the github site for most of the examples in the book.

Kruschke, John, Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press, 2014.

Reference text (recommended): This comprehensive text has become the standard reference in Bayesian statistical methods. The hyperlink below contains reviews, exercises, data sets and software. 

Gelman, A., Carlin, J., Stern, H., Dunson, D. B., Vehtari, A. and Rubin, D., Bayesian Data Analysis (3rd edition). CRC Press, 2013.

Supplemental text (recommended):  This recently published book provides comprehensive coverage of computational Bayesian statistics with a focus on conducting Bayesian analyses of real data sets.  The range of topics covered is much more extensive than the Hoff text, and will serve as a useful supplement for readers interested in Bayesian treatment of topics not covered in this course, such as generalized linear models, capture-recapture experiments, time series and image analysis.   R code and a solution manual are available.

Marin, Jean-Michel and Robert, Christian, Bayesian Essentials with R (2nd edition). Springer, 2014.

Alternate text
:
  The text by Peter Lee is accessible and may be helpful as an alternative treatment.   Again, the hyperlink contains additional information, including exercises, solutions, errata and software.
Lee, Peter, Bayesian Statistics:  An Introduction (4th edition), Wiley, 2012.

Software

Prerequisites

Requirements

Grades will be based on the following:

Homework assignments 30%

Midterm exam (take-home) 35%

Final exam (take-home) 35%

Eight to ten assignments will be given during the semester.  Assignments submitted before 23:59 the day after the due date will receive 75% credit.  Assignments submitted up to 1 week late receive 50% credit.  If you have extenuating circumstances, please contact me in advance, and I will consider giving you additional time to complete the assignment for partial credit. Assignments will be posted here and on Blackboard. Please submit your assignments through Gradescope.  Students are encouraged to work together on homework exercises, but solutions must be written up individually. Exams will be take-home and will be similar to the homework problems. Students are expected to work by themselves on the exams.

Policies and Resources

Schedule

The topics are listed below, along with readings from the Hoff text.  The take-home midterm exam will be posted by March 2 and will be due on Monday, March 21. The exam will include all material covered prior to March 14.  The final exam will be due on Monday, May 16 at 11:59 PM. The final exam will be cumulative. 


Unit 1 A Brief Tour of Bayesian Inference and Decision Theory
Week 1 Hoff, Chapter 1
Unit 2 Random variables, Parametric Models and Inference from Observation
Weeks 2-3 Hoff, Chapter 2
Unit 3 Bayesian Inference with Conjugate Pairs: Single Parameter Models Weeks 3-5 Hoff, Chapter 3
Unit 4 Introduction to Monte Carlo Approximation
Weeks 5-6 Hoff, Chapter 4
Unit 5 The Normal Model
Week 6-7 Hoff, Chapter 5
Unit 6 Markov Chain Monte Carlo
Week 8-9
Hoff, Chapters 6 and 9
Unit 7 Hierarchical Bayesian Models
Week 10-11 Hoff, Chapter 8
Unit 8 Bayesian Regression Week 12 Hoff, Chapter 9
Unit 9
Multinomial Distribution and Latent Groups
Week 13 Readings
Unit 10
Hypothesis Tests, Bayes Factors, and Bayesian Model Averaging
Week 14
Hoff, Chapter 10