Mean field variational inference

cosmos 10th April 2019 at 10:58am
Variational inference

A method of Variational inference where the space of distributions over which we optimize the variational distribution is distributions where the individual parameters are independent. This is known as a mean field approximation (as we ignore statistical dependences).

See here

In the above we show what the optimal qi(wi)q_i(w_i) is, when only optimizing qiq_i and keeping the rest of qjq_j fixed. This is a Coordinate descent update. So if this update is easy to calculate, we can use coordinate descent to optimize the mean field ELBO