Optimization Algorithms

Details of the available algorithms common to all of the optimizer tools.

There are two types of algorithms available in the Model Optimizer tool:

1. Local algorithms try to find the best solution in the neighborhood of a starting value. They search a narrower solution space, and so are more computationally efficient. They can yield more precise results, but may get stuck on local optima. They require an objective function that is smooth and differentiable.

2. Global algorithms search a large solution space to find the best overall solution. These algorithms are designed to avoid getting trapped in local minima or maxima. They are useful for complex, multi-modal landscapes with many local optima, but are more computationally expensive.

Local Algorithms

ADAM (Optax)

RMSProp (Optax)

Stochastic Gradient Descent (Optax)

L-BFGS-B (SciPy)

Sequential Least Squares (SciPy)

Global Algorithms

Particle Swarm Optimization (evosax)

Simulated Annealing (evosax)

CMA-ES (evosax)