A method to train Restricted Boltzmann machines

Contrastive divergence (CD)

Training Products of Experts by Minimizing Contrastive Divergence (Hinton) – On Contrastive Divergence Learning

contrastive divergence (parameter update)

Useful for pre-training (initializing) a neural networs; also for Feature selection (See here)

Persistent CD

Training restricted Boltzmann machines using approximations to the likelihood gradient – Using fast weights to improve persistent contrastive divergence

Neural networks [5.6] : Restricted Boltzmann machine - persistent CD

–>Motivation –> Idea of persistent CD

Results in much better performance in the negative log likelihood, thus providing a much better estimator of the distribution $p(x)$.