aka quadratic discriminant analysis

An example of a Generative supervised learning algorithm.

Lec vid intro, where we assume that $p(x|y)$ is Gaussian.

Definition (vid), for the case where $y \in \{0,1\}$

Learning by Maximum likelihood. The log likelihood (see Likelihood function), uses the joint likelihood $p(x,y|\theta)$. We maximize it to find the parameters. See more at Generative algorithm for the learning method.

Prediction, using Baye's theorem to get $p(y|x)$ and using the most likely $y$. See here.

Comparison with Logistic regression

See vid, vid2, vid3

The GDA makes stronger assumptions than Logistic regression. If the GDA Gaussian assumption holds, or roughly holds, GDA may do better than Logistic regression. In other cases, Logistic regression may do better.

Logistic regression is more flexible, but requires more data. GDA is less flexible, but can work well with less data if the stronger assumptions are correct.

Classification surface