Numerical Optimization

Your Progress in this Chapter

0%

0 / 12 units completed

Chapter Overview

NonLinear Optimization

Section 9.3 of 615 min6 code examples

Rosenbrook funcion with constraint

The goal is to find the parameter vector \mathbf{x} = [x_0, x_1]^T that minimizes the non - convex Rosenbrock objective function:

minxf(x0,x1)=100(x1x02)2+(1x0)2\min_{\mathbf{ x} } f(x_0, x_1) = 100(x_1 - x_0 ^ 2) ^ 2 + (1 - x_0) ^ 2

subject to the non-linear inequality constraint restricting the domain to the unit disk:

g(x)=x02+x1210g(\mathbf{ x}) = x_0 ^ 2 + x_1 ^ 2 - 1 \le 0
  • **Initial Guess : \mathbf{ x}_0 = (0, 0)

Example 1C#

1
2
3
4
Code is ready to run
OutputFrom the book
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found

Showing the first 80 of 504 lines.

Rosenbrook funcion with constraint, Lower and Upperbound

Example 2C#

1
2
3
4
5
6
7
8
9
10
11
Code is ready to run
OutputFrom the book
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
(
   0.5000
   0.2500
, 0.25, 1,   -0.0763, , System.Collections.Generic.List`1[SepalSolver.IterationState])

Rosenbrock function with constraint

The goal is to find the parameter vector :math:\mathbf{x} = [x_0, x_1]^T that minimizes the non-convex Rosenbrock objective function:

\min_{\mathbf{x}} f(x_0, x_1) = 100(x_1 - x_0^2)^2 + (1 - x_0)^2

subject to the non-linear inequality constraint restricting the domain to the unit disk:

g(\mathbf{x}) = x_0^2 + x_1^2 - 1 \le 0

  • Initial Guess: :math:\mathbf{x}_0 = (0, 0)

Example 3C#

1
2
3
4
Code is ready to run
OutputFrom the book
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Solving not completed
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found

Showing the first 80 of 504 lines.

Rosenbrock function with constraint, Lower and Upperbound

Minimizes the Rosenbrock objective subject to a shifted circular inequality constraint combined with explicit lower (lb) and upper (ub) parameter boundaries:

g(\mathbf{x}) = (x_0 - 0.333)^2 + (x_1 - 0.333)^2 - 0.11111 \le 0

0.0 \le x_0 \le 0.5, \quad 0.2 \le x_1 \le 0.8

Example 4C#

1
2
3
4
5
6
7
8
9
10
Code is ready to run
OutputFrom the book
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
Optimal solution found
(
   0.5000
   0.2500
, 0.25, 1,   -0.0763, , System.Collections.Generic.List`1[SepalSolver.IterationState])

Unconstrained Derivative-Free Optimization with Fminsearch

When gradient information is unavailable or the objective is non-differentiable, Fminsearch uses the Nelder-Mead Simplex algorithm to locate the unconstrained global minimum at :math:\mathbf{x}^ = (1, 1) where :math:f(\mathbf{x}^) = 0.

  • Initial Guess: :math:\mathbf{x}_0 = (-1.2, 1.0)

Example 5C#

1
2
3
4
5
6
7
Code is ready to run
OutputFrom the book
(
   1.0000
   1.0000
, 4.4768823537358065E-13, 1, , , System.Collections.Generic.List`1[SepalSolver.IterationState])

Global Stochastic Optimization with Genetic Algorithm

For non-convex or multimodal objective functions where gradient-based solvers risk getting trapped in local minima, the Genetic Algorithm (GA) uses population-based operators to explore bounded search spaces without requiring an initial guess.

-2.0 \le x_0 \le 2.0, \quad -2.0 \le x_1 \le 2.0

Example 6C#

1
2
3
4
5
6
7
8
9
10
11
12
Code is ready to run
OutputFrom the book
Stopping: no improvement for too long.
(
   0.9387
   0.8720
, False, 0.012145145679997998, 0x1 empty double row vector, 0x1 empty double row vector)