a function for which, given any two points, the function between those points sits at (lines are convex!) or below the plane given those points constituents For f: \mathbb{R}^{n} \to \mathbb{R} requirements \begin{equation} f\left(\theta x + \left(1-\theta\right) y\right) \leq \theta f\left(x\right) + \left(1-\theta\right) f\left(y\right) \end{equation} strictly convex is the strict inequality additional information log conditions f is log-linear IFF log f is affine f is log-concave iff log f is concave f is log-convex IFF log f is convex check if something is a convex function check definition restrict it to a line: convexity preserve line restriction 1st order condition 2nd order condition show that f is constructed by operations that preserve fuction convexity some convex functions affine: ax + b exponential: e^{ax} powers on R_{++}: x^{\alpha} for \alpha \geq 1 or \alpha \leq 0 |x|^{p} for p \geq 1 relu any norm sum of squares: \left\{x\right\}^{2}_{2} = x_1^{2} + … + x_{n}^{2} max function: \max \left(x\right) = \max \left\{x_1 \dots x_{n}\right\} softmax: \log \left(\exp x_1 + \dots + \exp x_{n}\right) general affine function: f\left(X\right) = tr\left(A^{T} X\right) + b (“an inner product”) spectral norm: f\left(X\right) = \norm{X}_{2} = \sigma_{\max}\left(X\right) (the maximum singular value of X logsumexp: f\left(x\right) = \log \sum_{k=1}^{n} \exp x_{k} quadratic over linear: f\left(x,y\right) = \frac{x^{2}}{y}, y >0 quadratic: f\left(x\right) = \frac{1}{2} x^{T} P x + q^{T} x + r, with P \succeq 0 is convex least squares: f\left(x\right) = \norm{Ax - b}^{2}_{2} is convex for any A inverse product: f\left(x\right) = \frac{1}{\prod_{i=1}^{n} x_{i}} inv_pos on R_{++}: f\left(x\right) = \frac{1}{x} is convex if x is concave and positive some concave fuctions affine square root fractional powers min logs: \log x entropy: - x \log x negative part (opposite relu) log determinant: f\left(X\right) = \log \text{det} X geometric mean: f\left(x\right) = \left(\prod_{k=1}^{n} x_{k}\right)^{\frac{1}{n}} an \mathbb{R}_{++}^{n} sublevel set \begin{equation} C_{\alpha} = \left{x \in \text{dom f} \mid f\left(x\right) \leq \alpha \right} \end{equation} sublevel sets of convex functions are convex sets (but converse is false) epigraph \begin{equation} \text{epi } f = \left{\left(x, t\right) \in \mathbb{R}^{n+1} \mid x \in \text{dom } f, f\left(x\right) \leq t\right} \end{equation} f is convex IFF epi f is a convex set “shaded area above the graph”