Loss function

cosmos 24th January 2019 at 11:19pm
Machine learning

aka risk, cost, loss; or its opposite, utility, reward,... depending on the area or context

A loss function refers to a Function that one wishes to minimize, in some decision theoretic framework, like in learning, Operations research, mathematical Ethics, etc., Optimization. Its oppositve (negative) is the Utility function, which one wishes to maximize.

Loss function for learning

Loss functions in Regression analysis

Video

Squared loss

Absolute value loss

Epsilon insensitive loss function

Loss functions in Classification

Video

Training error The problem is that this is not convex. That is why Hinge loss is better.

Hinge loss Video

Cross entropy

Negative log likelihood

Squared loss


See Empirical risk minimization

Expected loss, defined over Measurable functions

IIi
IIL