{ "cells": [ { "cell_type": "markdown", "id": "01374dd8", "metadata": {}, "source": [ "\n", "*This notebook contains material from [CBE60499](https://ndcbe.github.io/CBE60499);\n", "content is available [on Github](git@github.com:ndcbe/CBE60499.git).*\n" ] }, { "cell_type": "markdown", "id": "7f167674", "metadata": {}, "source": [ "\n", "< [4.2 Local Optimality Conditions](https://ndcbe.github.io/CBE60499/04.02-Local-Optimality.html) | [Contents](toc.html) | [Tag Index](tag_index.html) | [4.4 Constraint Qualifications](https://ndcbe.github.io/CBE60499/04.04-Constraint-Qualifications.html) >

\"Open

\"Download\"" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 1, "link": "[4.3 Analysis of KKT Conditions](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3-Analysis-of-KKT-Conditions)", "section": "4.3 Analysis of KKT Conditions" } }, "source": [ "# 4.3 Analysis of KKT Conditions" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 1, "link": "[4.3 Analysis of KKT Conditions](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3-Analysis-of-KKT-Conditions)", "section": "4.3 Analysis of KKT Conditions" } }, "source": [ "**Reference**: Section 4.3 in Biegler (2010)" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.1 Active Sets](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.1-Active-Sets)", "section": "4.3.1 Active Sets" } }, "source": [ "## 4.3.1 Active Sets" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.1 Active Sets](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.1-Active-Sets)", "section": "4.3.1 Active Sets" } }, "source": [ "![picture](./figures/active_constraints.png)" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.2 Sensitivity Analysis](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.2-Sensitivity-Analysis)", "section": "4.3.2 Sensitivity Analysis" } }, "source": [ "## 4.3.2 Sensitivity Analysis" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.2 Sensitivity Analysis](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.2-Sensitivity-Analysis)", "section": "4.3.2 Sensitivity Analysis" } }, "source": [ "![picture](./figures/active_constraints2.png)" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.2 Sensitivity Analysis](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.2-Sensitivity-Analysis)", "section": "4.3.2 Sensitivity Analysis" } }, "source": [ "![picture](./figures/def-4-6.png)" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.2 Sensitivity Analysis](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.2-Sensitivity-Analysis)", "section": "4.3.2 Sensitivity Analysis" } }, "source": [ "![picture](./figures/ex-4-7.png)\n", "\n", "![picture](./figures/ex-4-7b.png)" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.3 Multipliers in Pyomo](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3-Multipliers-in-Pyomo)", "section": "4.3.3 Multipliers in Pyomo" } }, "source": [ "## 4.3.3 Multipliers in Pyomo" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.3 Multipliers in Pyomo](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3-Multipliers-in-Pyomo)", "section": "4.3.3 Multipliers in Pyomo" } }, "source": [ "Reference: https://pyomo.readthedocs.io/en/stable/pyomo_modeling_components/Suffixes.html" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 2, "link": "[4.3.3 Multipliers in Pyomo](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3-Multipliers-in-Pyomo)", "section": "4.3.3 Multipliers in Pyomo" } }, "source": [ "$$\n", "\\begin{align} \\min_{x_1,...,x_4} \\quad & x_1 \\cdot x_4 \\cdot (x_1 + x_2 + x_3) + x_3 \\\\\n", "\\mathrm{s.t.} \\quad & x_1 \\cdot x_2 \\cdot x_3 \\cdot x_4 \\geq 25 \\\\\n", " & x_1 + x_2 + x_3 + x_4 = 40\n", "\\end{align}\n", "$$" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "nbpages": { "level": 2, "link": "[4.3.3 Multipliers in Pyomo](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3-Multipliers-in-Pyomo)", "section": "4.3.3 Multipliers in Pyomo" } }, "outputs": [], "source": [ "import pyomo.environ as pyo\n", "\n", "# Example from\n", "# https://pyomo.readthedocs.io/en/stable/pyomo_modeling_components/Suffixes.html#exporting-suffix-data\n", "\n", "model = pyo.ConcreteModel()\n", "model.x1 = pyo.Var(bounds=(1,5),initialize=1.0)\n", "model.x2 = pyo.Var(bounds=(1,5),initialize=5.0)\n", "model.x3 = pyo.Var(bounds=(1,5),initialize=5.0)\n", "model.x4 = pyo.Var(bounds=(1,5),initialize=1.0)\n", "model.obj = pyo.Objective(\n", " expr=model.x1*model.x4*(model.x1 + model.x2 + model.x3) + model.x3)\n", "model.inequality = pyo.Constraint(\n", " expr=model.x1*model.x2*model.x3*model.x4 >= 25.0)\n", "model.equality = pyo.Constraint(\n", " expr=model.x1**2 + model.x2**2 + model.x3**2 + model.x4**2 == 40.0)\n", "\n", "### Declare all suffixes\n", "# Ipopt bound multipliers (obtained from solution)\n", "model.ipopt_zL_out = pyo.Suffix(direction=pyo.Suffix.IMPORT)\n", "model.ipopt_zU_out = pyo.Suffix(direction=pyo.Suffix.IMPORT)\n", "# Ipopt bound multipliers (sent to solver)\n", "model.ipopt_zL_in = pyo.Suffix(direction=pyo.Suffix.EXPORT)\n", "model.ipopt_zU_in = pyo.Suffix(direction=pyo.Suffix.EXPORT)\n", "# Obtain dual solutions from first solve and send to warm start\n", "model.dual = pyo.Suffix(direction=pyo.Suffix.IMPORT_EXPORT)\n", "\n", "ipopt = pyo.SolverFactory('ipopt')" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.1 Solve without warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.1-Solve-without-warm-starting)", "section": "4.3.3.1 Solve without warm starting" } }, "source": [ "### 4.3.3.1 Solve without warm starting" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.1 Solve without warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.1-Solve-without-warm-starting)", "section": "4.3.3.1 Solve without warm starting" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ipopt 3.13.2: \n", "\n", "******************************************************************************\n", "This program contains Ipopt, a library for large-scale nonlinear optimization.\n", " Ipopt is released as open source code under the Eclipse Public License (EPL).\n", " For more information visit http://projects.coin-or.org/Ipopt\n", "******************************************************************************\n", "\n", "This is Ipopt version 3.13.2, running with linear solver ma27.\n", "\n", "Number of nonzeros in equality constraint Jacobian...: 4\n", "Number of nonzeros in inequality constraint Jacobian.: 4\n", "Number of nonzeros in Lagrangian Hessian.............: 10\n", "\n", "Total number of variables............................: 4\n", " variables with only lower bounds: 0\n", " variables with lower and upper bounds: 4\n", " variables with only upper bounds: 0\n", "Total number of equality constraints.................: 1\n", "Total number of inequality constraints...............: 1\n", " inequality constraints with only lower bounds: 1\n", " inequality constraints with lower and upper bounds: 0\n", " inequality constraints with only upper bounds: 0\n", "\n", "iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls\n", " 0 1.6109693e+01 1.12e+01 5.28e-01 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0\n", " 1 1.6982239e+01 7.30e-01 1.02e+01 -1.0 6.11e-01 - 7.19e-02 1.00e+00f 1\n", " 2 1.7318411e+01 3.60e-02 5.05e-01 -1.0 1.61e-01 - 1.00e+00 1.00e+00h 1\n", " 3 1.6849424e+01 2.78e-01 6.68e-02 -1.7 2.85e-01 - 7.94e-01 1.00e+00h 1\n", " 4 1.7051199e+01 4.71e-03 2.78e-03 -1.7 6.06e-02 - 1.00e+00 1.00e+00h 1\n", " 5 1.7011979e+01 7.19e-03 8.50e-03 -3.8 3.66e-02 - 9.45e-01 9.98e-01h 1\n", " 6 1.7014271e+01 1.74e-05 9.78e-06 -3.8 3.33e-03 - 1.00e+00 1.00e+00h 1\n", " 7 1.7014021e+01 1.23e-07 1.82e-07 -5.7 2.69e-04 - 1.00e+00 1.00e+00h 1\n", " 8 1.7014017e+01 1.77e-11 2.52e-11 -8.6 3.32e-06 - 1.00e+00 1.00e+00h 1\n", "\n", "Number of Iterations....: 8\n", "\n", " (scaled) (unscaled)\n", "Objective...............: 1.7014017145179160e+01 1.7014017145179160e+01\n", "Dual infeasibility......: 2.5166710821230136e-11 2.5166710821230136e-11\n", "Constraint violation....: 1.7706724975141697e-11 1.7706724975141697e-11\n", "Complementarity.........: 2.5277100427932999e-09 2.5277100427932999e-09\n", "Overall NLP error.......: 2.5277100427932999e-09 2.5277100427932999e-09\n", "\n", "\n", "Number of objective function evaluations = 9\n", "Number of objective gradient evaluations = 9\n", "Number of equality constraint evaluations = 9\n", "Number of inequality constraint evaluations = 9\n", "Number of equality constraint Jacobian evaluations = 9\n", "Number of inequality constraint Jacobian evaluations = 9\n", "Number of Lagrangian Hessian evaluations = 8\n", "Total CPU secs in IPOPT (w/o function evaluations) = 0.002\n", "Total CPU secs in NLP function evaluations = 0.000\n", "\n", "EXIT: Optimal Solution Found.\n" ] }, { "data": { "text/plain": [ "{'Problem': [{'Lower bound': -inf, 'Upper bound': inf, 'Number of objectives': 1, 'Number of constraints': 2, 'Number of variables': 4, 'Sense': 'unknown'}], 'Solver': [{'Status': 'ok', 'Message': 'Ipopt 3.13.2\\\\x3a Optimal Solution Found', 'Termination condition': 'optimal', 'Id': 0, 'Error rc': 0, 'Time': 0.05761003494262695}], 'Solution': [OrderedDict([('number of solutions', 0), ('number of solutions displayed', 0)])]}" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ipopt.solve(model, tee=True)" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.1 Solve without warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.1-Solve-without-warm-starting)", "section": "4.3.3.1 Solve without warm starting" } }, "source": [ "Inspect dual variables for lower bound" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.1 Solve without warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.1-Solve-without-warm-starting)", "section": "4.3.3.1 Solve without warm starting" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ipopt_zL_out : Direction=Suffix.IMPORT, Datatype=Suffix.FLOAT\n", " Key : Value\n", " x1 : 1.087871225865903\n", " x2 : 6.693166200639301e-10\n", " x3 : 8.887657145296478e-10\n", " x4 : 6.570872591662968e-09\n" ] } ], "source": [ "model.ipopt_zL_out.display()" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.1 Solve without warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.1-Solve-without-warm-starting)", "section": "4.3.3.1 Solve without warm starting" } }, "source": [ "Inspect dual variables for upper bound" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.1 Solve without warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.1-Solve-without-warm-starting)", "section": "4.3.3.1 Solve without warm starting" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ipopt_zU_out : Direction=Suffix.IMPORT, Datatype=Suffix.FLOAT\n", " Key : Value\n", " x1 : -6.262653086171725e-10\n", " x2 : -9.788835007044501e-09\n", " x3 : -2.12284925206338e-09\n", " x4 : -6.925197858855533e-10\n" ] } ], "source": [ "model.ipopt_zU_out.display()" ] }, { "cell_type": "markdown", "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.2 Solve with warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.2-Solve-with-warm-starting)", "section": "4.3.3.2 Solve with warm starting" } }, "source": [ "### 4.3.3.2 Solve with warm starting" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbpages": { "level": 3, "link": "[4.3.3.2 Solve with warm starting](https://ndcbe.github.io/CBE60499/04.03-KKT-Multipliers.html#4.3.3.2-Solve-with-warm-starting)", "section": "4.3.3.2 Solve with warm starting" } }, "outputs": [], "source": [ "### Set Ipopt options for warm-start\n", "# The current values on the ipopt_zU_out and ipopt_zL_out suffixes will\n", "# be used as initial conditions for the bound multipliers to solve the\n", "# new problem\n", "model.ipopt_zL_in.update(model.ipopt_zL_out)\n", "model.ipopt_zU_in.update(model.ipopt_zU_out)\n", "ipopt.options['warm_start_init_point'] = 'yes'\n", "ipopt.options['warm_start_bound_push'] = 1e-6\n", "ipopt.options['warm_start_mult_bound_push'] = 1e-6\n", "ipopt.options['mu_init'] = 1e-6\n", "\n", "ipopt.solve(model, tee=True)" ] }, { "cell_type": "markdown", "id": "d1a130e1", "metadata": {}, "source": [ "\n", "< [4.2 Local Optimality Conditions](https://ndcbe.github.io/CBE60499/04.02-Local-Optimality.html) | [Contents](toc.html) | [Tag Index](tag_index.html) | [4.4 Constraint Qualifications](https://ndcbe.github.io/CBE60499/04.04-Constraint-Qualifications.html) >

\"Open

\"Download\"" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.6" } }, "nbformat": 4, "nbformat_minor": 2 }