STA 4273H Fall 2012 - Lectures

Lecture Schedule

• Sept 11 -- Machine Learning:
  Introduction to Machine Learning, Linear Models for Regression (notes [pdf])
  Reading: Bishop, Chapter 1: sec. 1.1 - 1.5. and Chapter 3: sec. 1.1 - 1.3.
  Optional: Bishop, Chapter 2: Backgorund material;
  Hastie, Tibshirani, Friedman, Chapters 2 and 3.

• Sept 18 -- Bayesian Framework:
  Bayesian Linear Regression, Evidence Maximization. (notes [pdf])
  Reading: Bishop, Chapter 3: sec. 3.3 - 3.5.
  Optional: Radford Neal's NIPS tutorial on Bayesian Methods for Machine Learning: [pdf])

• Sep 27 -- Classification
  Linear Models for Classification, Generative and Discriminative approaches, Laplace Approximation. (notes [pdf])
  Reading: Bishop, Chapter 4.
  Optional: Hastie, Tibshirani, Friedman, Chapter 4.
  Also see Max Welling's notes on Fisher Linear Discriminant Analysis [pdf ]

• Oct 2 -- Graphical Models:
  Bayesian Networks, Markov Random Fields (notes [pdf])
  Reading: Bishop, Chapter 8.
  Optional: Hastie, Tibshirani, Friedman, Chapter 17 (Undirected Graphical Models).
  MacKay, Chapter 21 (Bayesian nets) and Chapter 43 (Boltzmann mchines).
  Also see this paper on Graphical models, exponential families, and variational inference by M. Wainwright and M. Jordan, Foundations and Trends in Machine Learning, [ here ]

• Oct 9 -- Mixture Models and EM:
  Mixture of Gaussians, Generalized EM, Variational Bound. (notes [pdf])
  Reading: Bishop, Chapter 9.
  Optional: Hastie, Tibshirani, Friedman, Chapter 13 (Prototype Methods).
  MacKay, Chapter 22 (Maximum Likelihood and Clustering).

• Oct 16 -- Variational Inference
  Mean-Field, Bayesian Mixture models, Variational Bound. (notes [pdf])
  Reading: Bishop, Chapter 10.
  Optional: MacKay, Chapter 33 (Variational Inference).

• Oct 23 - Sampling Methods
  Rejection Sampling, Importance sampling, M-H and Gibbs. (notes [pdf])
  Reading: Bishop, Chapter 11.
  Optional: MacKay, Chapter 29 (Monte Carlo Methods).

• Oct 30 -- Continuous Latent Variable Models
  PCA, FA, ICA, Deep Autoencders (notes [pdf])
  Reading: Bishop, Chapter 12.
  Optional: Hastie, Tibshirani, Friedman, Chapters 14.5, 14.7, 14.9 (PCA, ICA, nonlinear dimensionality reduction).
  MacKay, Chapter 34 (Latent Variable Models).

• Nov 6 -- Modeling Sequential Data
  HMMs, LDS, Particle Filters. (notes [pdf])
  Reading: Bishop, Chapter 13.
  

• Nov 20 -- Student Presentations

• Nov 27 -- Gaussian Processes, Combining Models.
  (notes [pdf])

STA 4273H (Fall 2012): Research Topics In Statistical Machine Learning || http://www.utstat.toronto.edu/~rsalakhu/sta4273_2012/