Learning Deep Boltzmann Machines
Code provided by Ruslan Salakhutdinov
Permission is granted for anyone to copy, use, modify, or distribute this
program and accompanying programs and documents for any purpose, provided
this copyright notice is retained and prominently displayed, along with
a note saying that the original programs are available from our
The programs and documents are distributed without any warranty, express or
implied. As the programs were written for research purposes only, they have
not been tested to the degree that would be advisable in any important
application. All use of these programs is entirely at the user's own risk.
How to make it work:
- Create a separate directory and download all these files into the same directory
- Download from http://yann.lecun.com/exdb/mnist
the following 4 files:
- Unzip these 4 files by executing:
If unzipping with WinZip, make sure the file names have not been
changed by WinZip.
- gunzip train-images-idx3-ubyte.gz
- gunzip train-labels-idx1-ubyte.gz
- gunzip t10k-images-idx3-ubyte.gz
- gunzip t10k-labels-idx1-ubyte.gz
- Download code_DBM.tar which contains 16 files
each of the following 16 files separately:
- demo_small.m Main file for training and fine-tuning a toy DBM model.
- demo.m Main file for training and fine-tuning a DBM model (reproduces results of the DBM paper).
- converter.m Converts raw MNIST digits into matlab format.
- rbm.m Training RBM with binary hidden and binary visible units.
- rbm_l2.m Training 2nd layer RBM with binary hidden and visible units.
- dbm_mf.m Joint training of all layers in a DBM.
- mf.m Implements mean-field inference.
- backprop.m Backpropagation for fine-tuning a DBM.
- mf_class.m Helper function used by backprop.m
- CG_MNIST.m Conjugate Gradient optimization for fine-tuning a DBM.
- CG_MNIST_INIT.m Conjugate Gradient optimization for fine-tuning a DBM (training top-level
parameters, while holding low-level parameters fixed).
- makebatches.m Creates minibatches for DBM training.
- testerr.m Computes misclassification error on the MNIST dataset.
- dispims.m Displays progress during DBM training.
- minimize.m Conjugate gradient code.
- For the toy experiment, run demo_small in matlab.
- Run demo in Matlab. This is a long version of training a Deep Boltzmann Machine, which
can take over 2 days. The code also uses sparsity to improve model performance.
- Make sure you have enough space to
store the entire MNIST dataset on your disk.
I did not try to optimize this code, but if you find bugs, please e-mail me.