Generative model

cosmos 29th November 2018 at 2:16pm
Probabilistic model Unsupervised learning

In general, a generative model is just a Probabilistic model. Technically, we call a probabilistic model a generative model, when used as the model of the "data-generating process" in the context of Unsupervised learning, or in Supervised learning. Here we will focus on their application in unsupervised learning (which is somehow more fundamental). For their application in supervised learning, see Generative supervised learning.

Generative modeling is a class of Unsupervised learning tasks, where given a data set of xxs, the task is to find/build a Generative model (Probabilistic model) P(x)P(x), for the training data. This model is then able to generate new data by sampling new xxs from PP; but it can also be used for many kinds of Inference of unknown quantities, including prediction as in Generative supervised learning.

Vid, can be used for Anomaly detection.

Mixture model

Gaussian mixture model

Factor analysis model

Density estimation

Often refers to non-parametric generative models. In fact, when one says generative model, one often refers to a parametric model for P(x)P(x).

Using Baye's theorem, generative models can be applied to Supervised learning, see Generative supervised learning

What is the relationship between generative models and density estimation?

wiki

Autoencoder

Generative adversarial network

GAN-visual analogy making

Pixel-RNN, codeRecurrent neural network

Autoregressive models. https://arxiv.org/pdf/1703.03664.pdf

Latent variables

Often, generative models have certain parameters which are particularly important, because they may represent a particular feature or structure that the generative model may be able to extract in its learning process. Generative models are in fact often designed with this in mind, as the task of Unsupervised learning is to find some interesting structure in the data.

See this video

https://openai.com/blog/generative-models/deploy@guillefix:~/cosmos.git/tiddlers

GENERATIVE TEMPORAL MODELS WITH MEMORY

Graphical models

http://science.sciencemag.org/content/358/6368/eaag2612.full?ijkey=DmvGldXIEXVoQ&keytype=ref&siteid=sci