Beginner Explanation
Imagine you are holding a rubber band between two points on a hill. If the rubber band always stays above the hill, then that hill is convex. This means that if you pick any two spots on the hill and draw a straight line between them, the line will never dip below the surface of the hill. So, a convex function is like a nice, smooth hill where everything is ‘uphill’ between any two points.Technical Explanation
A function f: R^n -> R is convex if for all x, y in R^n and for all t in [0, 1], the following holds: f(tx + (1-t)y) ≤ tf(x) + (1-t)f(y). This means that the line segment connecting any two points on the graph of f lies above or on the graph itself. In Python, you can check convexity using libraries like NumPy or SciPy. For example, you can use the Hessian matrix to determine convexity: if the Hessian is positive semi-definite for all points, the function is convex. Here’s a simple code snippet to check convexity of a quadratic function: “`python import numpy as np def hessian(f, x): h = 1e-5 n = len(x) hess = np.zeros((n, n)) for i in range(n): for j in range(n): x1 = np.copy(x) x1[i] += h x1[j] += h x2 = np.copy(x) x2[i] += h x2[j] -= h x3 = np.copy(x) x3[i] -= h x3[j] += h x4 = np.copy(x) x4[i] -= h x4[j] -= h hess[i, j] = (f(x1) – f(x2) – f(x3) + f(x4)) / (4 * h**2) return hess # Example function: f(x) = x^2 f = lambda x: x**2 x = np.array([1.0]) hess = hessian(f, x) print(hess) “`Academic Context
In mathematical optimization, convex functions are crucial because they guarantee that any local minimum is also a global minimum. Formally, a function f: R^n -> R is convex if for any x, y in R^n and t in [0, 1], the inequality f(tx + (1-t)y) ≤ tf(x) + (1-t)f(y) holds. This property leads to efficient algorithms for optimization problems, particularly in convex optimization. Key papers include ‘Convex Analysis’ by R. Tyrrell Rockafellar, which lays the groundwork for the theory of convex functions and optimization. The mathematical foundation relies on concepts such as subgradients, the Hessian matrix, and the notion of convex sets, which are essential in understanding the behavior of these functions in higher dimensions.Code Examples
Example 1:
import numpy as np
def hessian(f, x):
h = 1e-5
n = len(x)
hess = np.zeros((n, n))
for i in range(n):
for j in range(n):
x1 = np.copy(x)
x1[i] += h
x1[j] += h
x2 = np.copy(x)
x2[i] += h
x2[j] -= h
x3 = np.copy(x)
x3[i] -= h
x3[j] += h
x4 = np.copy(x)
x4[i] -= h
x4[j] -= h
hess[i, j] = (f(x1) - f(x2) - f(x3) + f(x4)) / (4 * h**2)
return hess
# Example function: f(x) = x^2
f = lambda x: x**2
x = np.array([1.0])
hess = hessian(f, x)
print(hess)Example 2:
h = 1e-5
n = len(x)
hess = np.zeros((n, n))
for i in range(n):
for j in range(n):
x1 = np.copy(x)
x1[i] += h
x1[j] += h
x2 = np.copy(x)
x2[i] += h
x2[j] -= h
x3 = np.copy(x)
x3[i] -= h
x3[j] += h
x4 = np.copy(x)
x4[i] -= h
x4[j] -= h
hess[i, j] = (f(x1) - f(x2) - f(x3) + f(x4)) / (4 * h**2)
return hessExample 3:
import numpy as np
def hessian(f, x):
h = 1e-5
n = len(x)Example 4:
def hessian(f, x):
h = 1e-5
n = len(x)
hess = np.zeros((n, n))
for i in range(n):View Source: https://arxiv.org/abs/2511.16568v1