A Function is convex over some if it satisfies the inequality

$f(\alpha x + \beta y) \leq \alpha f(x) + \beta f(y)$

for all $x, y \in D$ (where $D$ is the domain) and all $\alpha, \beta \in \mathbf{R}$ with $\alpha+\beta=1, \alpha \geq 0, \beta \geq 0$ (i.e. this is a Convex combination

The domain $D$ has to be a Convex set for this to be defined

Intuitively, a convex function is one for which the weighted average of the inputs produces at most the same weighted averages of the outputs.

Equivalent conditions for differentiable functions

• Once-differentiable. Tangent is lower bound. $g(y) + \nabla g(y)^T (x-y) \leq g(x)$.
• If the function is twice Differentiable, then it is convex iff $f'' \geq 0$. For multivariate, Hessian has to be positive semi-definite. This can be proved as shown in page 26 of Elements of info theory by Cover and Thomas. Needs the Cauchy form of Taylor's theorem

Relations to Convex sets

All the points above the function (the function's Epigraph) form a convex set (vid)

Also Level sets of convex functions are convex

Operations that preserve convexity

• Non-negative weighted sums of convex functions. If you allow negative weights, we get a problem of Difference of convex functions.
• Pointwise maximum of the functions.