Course policies and submitting this assignment¶
Read the Artificial Intelligence Policy and Collaboration Policy and Honor Code before starting. The assignment-specific directions below control where and when AI may be used.
Submit two files to Canvas:
A scanned PDF of your handwritten work for the pencil-and-paper problems. A clear phone photo assembled into a single PDF is fine.
A copy of this notebook, with your code cells run.
# This code cell installs packages on Colab
import sys
if "google.colab" in sys.modules:
!wget "https://raw.githubusercontent.com/ndcbe/optimization/main/notebooks/helper.py"
import helper
helper.easy_install()
else:
sys.path.insert(0, "../")
import helper
helper.set_plotting_style()## IMPORT LIBRARIES
import pyomo.environ as pyo
import pandas as pdHow this assignment is organized¶
This homework has three problems. Problems 1 and 2 are code. Problem 3 is pencil and paper --- no solver, no code cell --- and is deliberately written in the format of the in-person midterm, so that the exam is not the first time you see that kind of question.
| # | Problem | Format |
|---|---|---|
| 1 | Pyomo fundamentals: the knapsack problem | code |
| 2 | Lot sizing | code |
| 3 | Big- and convex hull reformulations | pencil and paper |
Problem 3 comes from Logical Modeling and Generalized Disjunctive Programming, and it is the first half of that thread. The second half --- writing a disjunction from an English specification, and handing both reformulations to Pyomo on a problem that is not small --- is Pyomo Homework 3, which refers back to your answers here.
Problem 1. Pyomo fundamentals: the knapsack problem¶
Parts 1-A through 1-I are all the same knapsack problem, developed from a first solve through to enumerating near-optimal solutions with integer cuts. Part 1-J is a short syntax exercise on a different model.
Problems 1 and 2 are adapted from the Pyomo team’s excellent PyomoFest workshop (Bynum et al., 2021). Special thanks to them for creating these exercises.
1-A. Knapsack example¶
You want to fill a knapsack (a.k.a. bag). You can choose from a hammer, wrench, screwdriver, and towel. Each item has a different weight and value. You want to maximize the value (benefit) of the collection of items constrained by a total weight limit. Let’s formulate this as an optimization problem.
Sets
Parameters (Data)
Let and represent the benefit and weight of item , respectively.
| Item () | Benefit () | Weight () |
|---|---|---|
| hammer | 8 | 5 |
| wrench | 3 | 7 |
| screwdriver | 6 | 4 |
| towel | 11 | 3 |
Let be the maximum weight.
Variables
Let (binary) represent whether or not we include item in the knapsack. For now, we will consider only being able to choose either none or one of each item.
Objective and Constraints
Pyomo
Solve the knapsack problem given below using HiGHS and answer the following questions:
Which items are acquired in the optimal solution?
Why does this solution make sense? (Write ~2 sentences.)
We use HiGHS, a modern open-source solver for linear and mixed-integer linear programs, which we call from Pyomo as pyo.SolverFactory('appsi_highs'). Earlier versions of this assignment used GLPK. HiGHS is faster, is actively developed, and installs anywhere with pip install highspy.
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
model.obj = pyo.Objective(expr=sum(b[i] * model.x[i] for i in A), sense=pyo.maximize)
model.weight_con = pyo.Constraint(expr=sum(w[i] * model.x[i] for i in A) <= W_max)
# Add your solution here
model.display()Question Answers
Fill in here
Fill in here
1-B. Knapsack example with improved printing¶
Complete the missing lines in the code below to produce formatted output: print the total weight, the value of the items selected (the objective), and the items acquired in the optimal solution. Note, the Pyomo value function should be used to get the floating point value of Pyomo modeling components (e.g., print(value(model.x[i])).
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
model.obj = pyo.Objective(expr=sum(b[i] * model.x[i] for i in A), sense=pyo.maximize)
model.weight_con = pyo.Constraint(expr=sum(w[i] * model.x[i] for i in A) <= W_max)
opt = pyo.SolverFactory("appsi_highs")
opt_success = opt.solve(model)
assert pyo.check_optimal_termination(opt_success), (
f"Solve failed: status={opt_success.solver.status}, "
f"termination={opt_success.solver.termination_condition}"
)
total_weight = sum(w[i] * pyo.value(model.x[i]) for i in A)
# Add your solution here
print("%12s %12s" % ("Item", "Selected"))
print("=========================")
for i in A:
acquired = "No"
# Add your solution here
print("-------------------------")1-C. Changing data¶
Using your code from 1-B, if we were to increase the value of the wrench, at what point would it become selected as part of the optimal solution?
# Add your solution hereQuestion Answer
Fill in here
1-D. Loading data from Excel¶
In the code above, the data is hardcoded at the top of the file. Instead of hardcoding the data, use Python to load the data from a different source. You may use Pandas to load data from ‘knapsack_data.xlsx’ into a dataframe. You will then need to write code to obtain a dictionary from the dataframe.
df_items = pd.read_excel(
"https://raw.githubusercontent.com/ndcbe/optimization/main/notebooks/data/knapsack_data.xlsx", sheet_name="data", header=0, index_col=0
)
W_max = 14
A = df_items.index.tolist()
# Add your solution here
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
model.obj = pyo.Objective(expr=sum(b[i] * model.x[i] for i in A), sense=pyo.maximize)
model.weight_con = pyo.Constraint(expr=sum(w[i] * model.x[i] for i in A) <= W_max)
opt = pyo.SolverFactory("appsi_highs")
opt_success = opt.solve(model, tee=True)
assert pyo.check_optimal_termination(opt_success), (
f"Solve failed: status={opt_success.solver.status}, "
f"termination={opt_success.solver.termination_condition}"
)
total_weight = sum(w[i] * pyo.value(model.x[i]) for i in A)
print("Total Weight:", total_weight)
print("Total Benefit:", pyo.value(model.obj))
print("%12s %12s" % ("Item", "Selected"))
print("=========================")
for i in A:
acquired = "No"
if pyo.value(model.x[i]) >= 0.5:
acquired = "Yes"
print("%12s %12s" % (i, acquired))
print("-------------------------")1-E. NLP vs. MIP¶
Solve the knapsack problem with IPOPT instead of HiGHS. Print the solution values for model.x. What happened? Why?
Hint: Switch appsi_highs to ipopt in the call to SolverFactory.
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
model.obj = pyo.Objective(expr=sum(b[i] * model.x[i] for i in A), sense=pyo.maximize)
model.weight_con = pyo.Constraint(expr=sum(w[i] * model.x[i] for i in A) <= W_max)
# Add your solution here
opt_success = opt.solve(model, tee=True)
assert pyo.check_optimal_termination(opt_success), (
f"Solve failed: status={opt_success.solver.status}, "
f"termination={opt_success.solver.termination_condition}"
)
model.pprint()Question Answers
Fill in here
1-F. Knapsack problem with rules¶
Rules are important for defining indexed constraints, however, they can also be used for single (i.e. scalar) constraints. Reimplement the knapsack model from 1-A using rules for the objective and the constraints.
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
# Add your solution here1-G. Integer formulation of the knapsack problem¶
Consider again the knapsack problem. Assume now that we can acquire multiple items of the same type. In this new formulation, is now an integer variable instead of a binary variable. One way to formulate this problem is as follows:
One could optionally add the following constraint to select only one for each , although it is not strictly necessary to yield an integer solution. $$
$$
Starting with your code from 1-F, implement this new formulation and solve. Is the solution surprising?
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
N = range(6) # create a list from 0-5
model = pyo.ConcreteModel()
model.x = pyo.Var(A)
model.q = pyo.Var(A, N, domain=pyo.Binary)
def obj_rule(m):
return sum(b[i] * m.x[i] for i in A)
model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize)
def weight_con_rule(m):
return sum(w[i] * m.x[i] for i in A) <= W_max
model.weight_con = pyo.Constraint(rule=weight_con_rule)
# Add your solution hereQuestion Answer
Fill in here
1-H. Changing parameter values with a mutable Param¶
A parameter can be specified to be mutable. This tells Pyomo that the value of the parameter may change in the future, and allows the user to change the parameter value and resolve the problem without the need to rebuild the entire model each time. We will use this functionality to find a better solution to the knapsack problem. We would like to find when the wrench becomes valuable enough to be a part of the optimal solution. Create a Pyomo Parameter for the value of the items, make it mutable, and then write a loop that prints the solution for different wrench values.
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
# Add your solution here
def obj_rule(m):
return sum(m.item_benefit[i] * m.x[i] for i in A)
model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize)
def weight_rule(m):
return sum(w[i] * m.x[i] for i in A) <= W_max
model.weight = pyo.Constraint(rule=weight_rule)
# You may instead use 'cbc' as the solver
opt = pyo.SolverFactory("appsi_highs")
for wrench_benefit in range(1, 11):
model.item_benefit["wrench"] = wrench_benefit
result_obj = opt.solve(model)
assert pyo.check_optimal_termination(result_obj), (
f"Solve failed: status={result_obj.solver.status}, "
f"termination={result_obj.solver.termination_condition}"
)
# Add your solution here1-I. Integer cuts¶
Often, it can be important to find not only the “best” solution, but a number of solutions that are equally optimal, or close to optimal. For discrete optimization problems, this can be done using something known as an integer cut. Consider again the knapsack problem where the choice of which items to select is a discrete variable . Let be a particular set of values we want to remove from the feasible solution space. We define an integer cut using two sets. The first set contains the indices for those variables whose current solution is 0, and the second set consists of indices for those variables whose current solution is 1. Given these two sets, an integer cut constraint that would prevent such a solution from appearing again is defined by,
Write a loop that solves the problem 5 times, adding an integer cut to remove the previous solution, and printing the value of the objective function and the solution at each iteration of the loop.
A = ["hammer", "wrench", "screwdriver", "towel"]
b = {"hammer": 8, "wrench": 3, "screwdriver": 6, "towel": 11}
w = {"hammer": 5, "wrench": 7, "screwdriver": 4, "towel": 3}
W_max = 14
model = pyo.ConcreteModel()
model.x = pyo.Var(A, domain=pyo.Binary)
def obj_rule(m):
return sum(b[i] * m.x[i] for i in A)
model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize)
def weight_con_rule(m):
return sum(w[i] * m.x[i] for i in A) <= W_max
model.weight_con = pyo.Constraint(rule=weight_con_rule)
# You may instead use 'cbc' as the solver
opt = pyo.SolverFactory("appsi_highs")
# create the ConstraintList to hold the integer cuts
model.int_cuts = pyo.ConstraintList()
# Add your solution here1-J. Decorator notation¶
Alternative notation for declaring and defining Pyomo components using decorators exists. Starting with the warehouse location problem code below, change the model to use the decorator notation.
This is the last part of Problem 1 and the only one that is not the knapsack problem: the point is the syntax, not the model. Problem 2 of Pyomo Homework 3 builds its model entirely with decorators, including @model.Disjunction, so this is the notation you will need there.
# warehouse_location.py: Warehouse location determination problem
model = pyo.ConcreteModel(name="(WL)")
W = ["Harlingen", "Memphis", "Ashland"]
C = ["NYC", "LA", "Chicago", "Houston"]
d = {
("Harlingen", "NYC"): 1956,
("Harlingen", "LA"): 1606,
("Harlingen", "Chicago"): 1410,
("Harlingen", "Houston"): 330,
("Memphis", "NYC"): 1096,
("Memphis", "LA"): 1792,
("Memphis", "Chicago"): 531,
("Memphis", "Houston"): 567,
("Ashland", "NYC"): 485,
("Ashland", "LA"): 2322,
("Ashland", "Chicago"): 324,
("Ashland", "Houston"): 1236,
}
P = 2
model.x = pyo.Var(W, C, bounds=(0, 1))
model.y = pyo.Var(W, domain=pyo.Binary)
@model.Objective()
def obj(m):
return sum(d[w, c] * m.x[w, c] for w in W for c in C)
@model.Constraint(C)
def one_per_cust(m, c):
return sum(m.x[w, c] for w in W) == 1
# Add your solution here
def warehouse_active(m, w, c):
return m.x[w, c] <= m.y[w]
# Note: This is only split across cells because of a bug in nbpages (notebook/website software).
# There is no other reason to split your code across cells.# Add your solution here
def num_warehouses(m):
return sum(m.y[w] for w in W) <= P
results = pyo.SolverFactory("appsi_highs").solve(model)
assert pyo.check_optimal_termination(results), (
f"Solve failed: status={results.solver.status}, "
f"termination={results.solver.termination_condition}"
)
model.y.pprint()
model.x.pprint()Problem 1 AI use report¶
At the end of Problem 1, report for the problem as a whole:
approximately how long you spent on the single 30-minute independent attempt for Problem 1;
how far you got during that independent attempt;
where you got stuck, if anywhere;
any AI or collaborative help you used afterward; and
how you verified that help.
If you used no AI, say so explicitly. A few concise bullets are sufficient; do not submit prompts or transcripts. The time estimate gives the instructor useful data for improving the assignment and is not a speed test. Apply the 30-minute guideline once to the entire problem, not separately to each subpart.
Problem 2. Lot sizing (Bynum et al., 2021)¶
We will now write a complete model from scratch using a well-known multi-period optimization problem for optimal lot-sizing adapted from Haugen et al. (2001) shown below.
s.t.
Our goal is to find the optimal production given known demands , fixed cost associated with active production in a particular time period, an inventory holding cost and a shortage cost (cost of keeping a backlog) of orders. The variable (binary) determines if we produce in time or not, and represents inventory that we are storing across time period , while represents the magnitude of the backlog. Note that is a constraint that only allows production in time period if the indicator variable =1.
Write a Pyomo model for this problem and solve it using HiGHS, i.e., pyo.SolverFactory('appsi_highs'), (or cbc) using the data provided below.
| Parameter | Description | Value |
|---|---|---|
| fixed cost of production | 4.6 | |
| initial value of positive inventory | 5.0 | |
| initial value of backlogged orders | 0.0 | |
| cost (per unit) of holding inventory | 0.7 | |
| shortage cost (per unit) | 1.2 | |
| maximum production amount (big-M value) | 5 | |
| demand | [5, 7, 6.2, 3.1, 1.7] |
Reference: Bynum, M. L., Hackebeil, G. A., Hart, W. E., Laird, C. D., Nicholson, B. L., Siirola, J. D., Watson, J.-P., and Woodruff, D. L. Pyomo — Optimization Modeling in Python, Third Edition. Springer Optimization and Its Applications, Vol. 67, 2021. (§8.6, p. 117)
model = pyo.ConcreteModel()
model.T = pyo.RangeSet(5) # time periods
i0 = 5.0 # initial inventory
c = 4.6 # setup cost
h_pos = 0.7 # inventory holding cost
h_neg = 1.2 # shortage cost
P = 5.0 # maximum production amount
# demand during period t
d = {1: 5.0, 2: 7.0, 3: 6.2, 4: 3.1, 5: 1.7}
# Add your solution here
# solve the problem
# You may instead use 'cbc' as the solver
solver = pyo.SolverFactory("appsi_highs")
results = solver.solve(model)
assert pyo.check_optimal_termination(results), (
f"Solve failed: status={results.solver.status}, "
f"termination={results.solver.termination_condition}"
)
# print the results
for t in model.T:
print("Period: {0}, Prod. Amount: {1}".format(t, pyo.value(model.x[t])))Problem 2 AI use report¶
At the end of Problem 2, report for the problem as a whole:
approximately how long you spent on the single 30-minute independent attempt for Problem 2;
how far you got during that independent attempt;
where you got stuck, if anywhere;
any AI or collaborative help you used afterward; and
how you verified that help.
If you used no AI, say so explicitly. A few concise bullets are sufficient; do not submit prompts or transcripts. The time estimate gives the instructor useful data for improving the assignment and is not a speed test. Apply the 30-minute guideline once to the entire problem, not separately to each subpart.
Problem 3. Big- and convex hull reformulations¶
Pencil and paper. No solver, no code cell. This problem is written in the format of the in-person midterm.
Background. A process must deliver a flow rate [kmol/h]. Exactly one of three pump types is installed, and each type has its own operating window and its own installed cost [thousand USD]:
| Pump | Operating window [kmol/h] | Installed cost [thousand USD] |
|---|---|---|
| 1 | 12 | |
| 2 | 20 | |
| 3 | 35 |
The windows are hard physical limits, not preferences: outside its window a pump cannot run at all. Independently of which pump is installed, the flow rate is bounded by kmol/h.
Instruction. Use the notation of the Logical Modeling and Generalized Disjunctive Programming lecture throughout. Boolean indicators are capital ; the binary variables that represent them are lower-case ; and every inequality is written in the standard form , so is written .
3-A. Write the pump selection as a disjunction¶
Write the selection in generalized disjunctive programming standard form. Give the index set, the Boolean indicators, the constraints inside each disjunct, the cost assignment, and the logical constraint .
Next to each set, indicator, and equation, write a few-word description.
Sets.
Parameters (Data). Hint: state the units of each one.
Variables.
The disjunction.
.
Then answer in one sentence: why can this not be written as a single pair of bounds on ?
3-B. Big- reformulation¶
Write the big- reformulation of your disjunction. Introduce a binary for each Boolean and write out all six relaxed inequalities explicitly, plus the constraint that selects one pump.
Determine the smallest valid for each of the three terms, using one per term. Show the reasoning, not just the number. Hint: must dominate the largest violation the term’s constraints can attain anywhere in .
In one or two sentences each: what goes wrong if is chosen too large, and what goes wrong if it is chosen too small? The two failures are not the same kind of failure.
You could instead use a separate for every inequality rather than one per term. Would that be tighter, looser, or the same? Give the six values.
3-C. Convex hull reformulation¶
Write the disaggregated convex hull reformulation. Disaggregate into , , --- one copy per pump --- and write out all the constraints explicitly.
The general form carries an optional bound . Is it needed here? Justify your answer by setting and reading off what the term’s own constraints force.
Note that the cost needs no big- and no disaggregation. Write , the installed cost, as a single linear equation in the , and explain in one sentence why that equation is exact rather than a relaxation.
3-D. Compare the two relaxations¶
This is the part that explains why anyone pays for the extra variables in 3-C.
Relax the integrality ( becomes ) and fix . Using your smallest valid per-term from 3-B, compute the interval of that the big- relaxation admits, and the interval that the convex hull relaxation admits. Show the arithmetic.
Which interval is contained in the other? Is the containment strict?
Now let range freely over with . What is the full projection onto of each relaxation --- that is, the set of values that survive for some fractional ? Compare each answer to the convex hull of , which is what the convex hull reformulation is supposed to give you.
One of the two answers in part 3 should alarm you. Say in two sentences what it means for branch and bound.
3-E. Problem size¶
Let be the number of disjunction terms, let be the continuous variables appearing inside the disjuncts, and let term carry inequality constraints, with . Using these symbols, determine the size of each reformulation. After each header below, give a number with a brief justification, for big- and for the convex hull separately.
Number of continuous variables:
Number of integer/discrete variables:
Number of equality constraints:
Number of inequality constraints:
Then fill in the numbers for the pump problem specifically (, , ).
3-F. Short answer¶
Two or three sentences each, not a short essay.
You must hand this model to two different audiences: a colleague who has to understand what the plant does, and a solver. Which of the three forms (the disjunction, big-, the convex hull) goes to each, and why?
Under what circumstances would you deliberately choose big- even though 3-D shows its relaxation is worse?
Problem 3 AI use report¶
At the end of Problem 3, report for the problem as a whole:
approximately how long you spent on the single 30-minute independent attempt for Problem 3;
how far you got during that independent attempt;
where you got stuck, if anywhere;
any AI or collaborative help you used afterward; and
how you verified that help.
If you used no AI, say so explicitly. A few concise bullets are sufficient; do not submit prompts or transcripts. The time estimate gives the instructor useful data for improving the assignment and is not a speed test. Apply the 30-minute guideline once to the entire problem, not separately to each subpart.