
STA 4273H Fall 2011  Lectures
Lecture Schedule
 Sept 13  Machine Learning:
Introduction to Machine Learning, Linear Models for Regression
(notes
[pdf])
Reading: Bishop, Chapter 1: sec. 1.1  1.5.
and Chapter 3: sec. 1.1  1.3.
Optional: Bishop, Chapter 2: Backgorund material;
Hastie, Tibshirani, Friedman, Chapters 2 and 3.
 Sept 20  Bayesian Framework:
Bayesian Linear Regression, Evidence Maximization.
(notes
[pdf])
Reading: Bishop, Chapter 3: sec. 3.3  3.5.
Optional:
Radford Neal's NIPS tutorial on
Bayesian Methods for Machine Learning:
[pdf])
 Sep 27  Classification
Linear Models for Classification, Generative and
Discriminative approaches, Laplace Approximation.
(notes
[pdf])
Reading: Bishop, Chapter 4.
Optional:
Hastie, Tibshirani, Friedman, Chapter 4.
Also see Max Welling's notes on
Fisher Linear Discriminant Analysis
[pdf]
 Oct 4  Graphical Models:
Bayesian Networks, Markov Random Fields
(notes
[pdf])
Reading: Bishop, Chapter 8.
Optional:
Hastie, Tibshirani, Friedman, Chapter 17 (Undirected Graphical Models).
MacKay, Chapter 21 (Bayesian nets) and Chapter 43 (Boltzmann mchines).
Also see this paper on
Graphical models, exponential families, and variational inference by
M. Wainwright and M. Jordan, Foundations and Trends in Machine Learning,
[ here
]
 Oct 11  Inference in Graphical Models:
Factor graphs, sumproduct algorithm, loopy belief propagation
(notes
[pdf])
Reading: Bishop, Chapter 8.
Optional:
Hastie, Tibshirani, Friedman, Chapter 17 (Undirected Graphical Models).
MacKay, Chapter 26 (Exact Marginalization in Graphs).
 Oct 18  Mixture Models and EM:
Mixture of Gaussians, Generalized EM, Variational Bound.
(notes
[pdf])
Reading: Bishop, Chapter 9.
Optional:
Hastie, Tibshirani, Friedman, Chapter 13 (Prototype Methods).
MacKay, Chapter 22 (Maximum Likelihood and Clustering).
 Oct 25  Variational Inference
MeanField, Bayesian Mixture models, Variational Bound.
(notes
[pdf])
Reading: Bishop, Chapter 10.
Optional:
MacKay, Chapter 33 (Variational Inference).
 Nov 1  Sampling Methods
Rejection Sampling, Importance sampling, MH and Gibbs.
(notes
[pdf])
Reading: Bishop, Chapter 11.
Optional:
MacKay, Chapter 29 (Monte Carlo Methods).
 Nov 15  Learning Hierarchical Models
Deep Belief Nets, Deep Boltzmann machines, Hierarchical LDA.
(notes
[pdf])
 Nov 22  Continuous Latent Variable Models
PCA, FA, ICA, Deep Autoencders
(notes
[pdf])
Reading: Bishop, Chapter 12.
Optional:
Hastie, Tibshirani, Friedman, Chapters 14.5, 14.7, 14.9 (PCA, ICA,
nonlinear dimensionality reduction).
MacKay, Chapter 34 (Latent Variable Models).
 Nov 29  Modeling Sequential Data
HMMs, LDS, Particle Filters. Project Presentations
(notes
[pdf])
Reading: Bishop, Chapter 13.
 Dec 6  Combining Models
Bagging, Boosting, Conclusions
(notes
[pdf])
Reading: Bishop, Chapter 14.
Optional:
Hastie, Tibshirani, Friedman, Chapter 10.
[
Home 
Course Information 
Lecture Schedule/Notes 
Assignments/Project 
Computing 
]
STA 4273H (Fall 2011): Research Topics In Statistical Machine Learning
 http://www.utstat.toronto.edu/~rsalakhu/sta4273/
