Beginner Explanation
Imagine you’re in a hilly landscape, and you’re trying to find the lowest point. A global minimum is like a small valley where you can see that it’s lower than the surrounding hills, but there might be even deeper valleys elsewhere that you can’t see from where you are. So, while you’re in a nice low spot, it might not be the absolute lowest spot in the entire landscape.Technical Explanation
In optimization problems, a global minimum refers to a point in the function where the function’s value is lower than at all other points in its domain. However, local minima are points where the function value is lower than its immediate neighbors, which may not be the lowest overall. In machine learning, algorithms like gradient descent are used to find minima. Here’s a simple Python example using NumPy to illustrate finding a local minimum: “`python import numpy as np # Define a simple quadratic function def f(x): return (x – 2) ** 2 + 1 # Gradient descent algorithm to find local minimum x = 0 # Start point learning_rate = 0.1 for _ in range(20): x -= learning_rate * 2 * (x – 2) # Gradient descent step print(f’Local minimum at x: {x}, f(x): {f(x)}’) “` This code finds a local minimum of the function, which is not guaranteed to be the global minimum unless the function is convex.Academic Context
The concept of global minima is foundational in optimization theory, particularly in the context of convex functions. A function is convex if any line segment connecting two points on the graph of the function lies above the graph itself. In such cases, any local minimum is also a global minimum. Key papers include ‘Convex Optimization’ by Boyd and Vandenberghe, which discusses conditions under which global minima can be efficiently found. The mathematical foundation often involves calculus, particularly the use of derivatives to identify critical points where the gradient is zero. Researchers also explore methods like stochastic gradient descent and evolutionary algorithms to navigate complex landscapes with multiple local minima, as discussed in ‘Neural Networks for Pattern Recognition’ by Christopher Bishop.Code Examples
Example 1:
import numpy as np
# Define a simple quadratic function
def f(x):
return (x - 2) ** 2 + 1
# Gradient descent algorithm to find local minimum
x = 0 # Start point
learning_rate = 0.1
for _ in range(20):
x -= learning_rate * 2 * (x - 2) # Gradient descent step
print(f'Local minimum at x: {x}, f(x): {f(x)}')Example 2:
return (x - 2) ** 2 + 1Example 3:
x -= learning_rate * 2 * (x - 2) # Gradient descent stepExample 4:
import numpy as np
# Define a simple quadratic function
def f(x):
return (x - 2) ** 2 + 1Example 5:
def f(x):
return (x - 2) ** 2 + 1
# Gradient descent algorithm to find local minimum
x = 0 # Start pointView Source: https://arxiv.org/abs/2511.15377v1