Linear Algebra and Linear Programming Notes

Linear Algebra and Linear Programming

Duality Theory

Primal and dual problems of LPs

如何理解?

\[\bar{x}=\left(\begin{array}{l} x \\ y \end{array}\right), \quad A=\left(\begin{array}{ll} 5 & 3 \\ 2 & 4 \end{array}\right), \quad b=\left(\begin{array}{c} 100 \\ 60 \end{array}\right), \quad c=\left(\begin{array}{l} 16 \\ 10 \end{array}\right) \]

供应商卖🍬和维生素C的单价
\(\rightarrow\)
\(u\) \(v\)
买几个水果\(\downarrow\) 🍬 维生素C 水果价格\(\downarrow\)
\(x\) 苹果 5 3 16
\(y\) 香蕉 2 4 10
需要的量 100 60

\[\text { Minimise } 16 x+10 y \text { (the total cost) } \]

消费者希望买到花最少的钱买到满足需求的产品;

\[\text { Maximise } 100 u+60 v \text { (the total profit) } \]

供应商希望在卖🍬和维生素C的时候赚最多的钱。

Primal Problem:

\[\begin{array}{cl} (L P) \quad \text { Minimize } & c^{T} x \\ \text { s.t. } & A x=b, \quad x \geq 0 \end{array} \]

Its dual problem is defined by
Dual problem:

\[\begin{array}{cl} (D P) \quad \text { Maximize } & b^{T} y \\ \text { s.t. } \quad & A^{T} y \leq c \end{array} \]

where \(y \in \mathbb{R}^{m}\).

消费者角度:最便宜的价格买符合要求的东西;

供应商角度:符合消费者需求的东西卖到最贵;

If the primal problem is given by (canonical form)

\[\begin{array}{cl} (L P) \quad \text { Minimize } & c^{T} x \\ \text { s.t. } & A x \geq b, \quad x \geq 0 \end{array} \]

then its dual problem is

\[\begin{array}{cl} (D P) \quad \text { Maximize } & b^{T} y \\ \text { s.t. } \quad & A^{T} y \leq c, y \geq 0 \end{array} \]

Duality theorem

6.7 (Weak duality Theorem). Let \(x\) be any feasible point of the primal problem, i.e. \(x \in F_{p}\) and \(y\) be any feasible point of the dual problem, i.e., \((y, s) \in F_{d}\). Then

\[c^{T} x \geq b^{T} y \]

消费者买的肯定比供货商的最低价格高;

\[c^T x − b^T y \]

is often called the duality gap.

Priamal 和 Dual 有一个unbound另一个也unbound.

如果$$c^T x = b^T y$$, 那\(x\)\(y\)都是最优解。

Theorem 6.12 (Strong Duality Theorem). If both the primal LP and the dual LP have feasible solutions, then they both have optimal solutions, and for any primal optimal solution \(x\) and dual optimal solution \(y\) we have that \(c^{T} x=b^{T} y .\)

如果都是最优解,那就「没有中间商赚差价」,消费者最便宜买到的跟供货商卖的是一样价格。

P optimal \(\iff\) D optimal

P unbounded \(\Rightarrow\) D infeasible

D unbounded \(\Rightarrow\) P infeasible

P infeasible \(\Rightarrow\) D unbounded or infeasible

D infeasible \(\Rightarrow\) P unbounded or infeasible

Network flow problems

最大流

最小割

Optimality Condition

Theorem 6.19 (Optimality Condition). Consider the standard LP problem:

\[\begin{array}{cl} \text { Minimize } & c^{T} x \\ \text { s.t. } & A x=b, \quad x \geq 0 \end{array} \]

where \(b \in R^{m}, c \in R^{n}, A \in R^{m \times n} .\) Then \(\bar{x}\) is an optimal solution to the \(L P\) if and only if there exists \(\bar{y}\) such that the following three conditions hold

\[\begin{aligned} A^{T} \bar{y} & \leq c \\ A \bar{x} &=b, \quad \bar{x} \geq 0 \\ b^{T} \bar{y} &=c^{T} \bar{x} \end{aligned} \]

  • \(\bar x, \bar y\) are feasible solutions
  • no duality gap

Proposition 6.20 (Reformulation of optimality conditions). The optimality conditions \((6.2)-(6.4)\), i.e.,

\[\begin{aligned} A^{T} \bar{y} & \leq c \\ A \bar{x} &=b, \quad \bar{x} \geq 0 \\ b^{T} \bar{y} &=c^{T} \bar{x} \end{aligned} \]

can be restated as follows

\[\begin{aligned} A^{T} \bar{y}+\bar{s} &=c, \quad \bar{s} \geq 0 \\ A \bar{x} &=b, \quad \bar{x} \geq 0 \\ \bar{x}^{T} \bar{s} &=0 \end{aligned} \]

Complementary slackness condition

\(x^*\) and \((y^*,s^*)\) are optimal solutions to primal and dual problems, respectively, if and only if they satisfy the primal feasibility, dual feasibility and the complemen-tary slackness condition.

\[\begin{aligned} A^{T} \bar{y}+\bar{s} &=c, & \bar{s} \geq 0 . & & \text { (Dual feasibility) } \\ A \bar{x} &=b, & \bar{x} \geq 0 . & & \text { (Primal feasibility) } \\ \bar{x}^{T} \bar{s} &=0 . & & & \text { (Complementary slackness condition) } \end{aligned} \]

\(x^*\)\(s^*\)至少有一个是\(0\)

如果Dual problem中对应的条件是严格不等的,\(x^*_i = 0\),

如果\(x^*_i \neq 0\), D中对应的条件退化成等式;

The Simplex Method

Theorem 5.19 (Minkowski's Theorem). If \(P=\left\{x \in R^{n}: A x \leq b\right\}\) is nonempty and \(\operatorname{rank}(A)=n\), then

\[\begin{aligned} P=\{x: x&=\sum_{j=1}^{k} \lambda_{j} x^{j}+\sum_{i=1}^{l} \mu_{i} d^{i}, \sum_{j=1}^{k} \lambda_{j}=1 \\ \lambda_{j} &\left.\geq 0, j=1, \ldots, k, \mu_{i} \geq 0, i=1, \ldots, l\right\} \end{aligned} \]

where \(\left\{x^{1}, x^{2}, \ldots, x^{k}\right\}\) is the set of extreme points of \(P\) and \(\left\{d^{1}, d^{2}, \ldots, d^{l}\right\}\) is the set of extreme rays of \(P\).

\(n\)维的空间,可行域约束条件式子是线性独立的\(n\)个,任何一个可行域中的点可以表示为 extreme point 和 extreme direction 的线性组合.

Theorem 7.2 (Existence of Optimal Extreme Point). Assume that the feasible region is nonempty. Then

  • an (finite) optimal solution exists if and only if \(c^{T} d^{j} \geq 0\) for \(j=1, \ldots, l\), where \(d^{1}, \ldots, d^{l}\) are the extreme directions of the feasible region.
  • Otherwise, if there is an extreme direction such that \(c^{T} d^{j}<0\), then the optimal objective value of \(L P\) is unbounded.
  • If an optimal solution exists, then at least one extreme point is optimal.

只有\(c, d\)方向相同的时候才可能有feasible solution, 否则只要\(x\)无穷大,目标值就无穷小.

Basic Feasible Solution

解能解的部分,剩下的位放\(0\).

Correspondence between BFSs & extreme points

A point is a basic feasible solution if and only if it is an extreme point.

BFS & extreme point 是一个东西;

Theorem 7.6 (Fundamental theorem of Linear Programming).
i) If there is a feasible solution, i.e., \(\{x: A x=b, x \geq 0\} \neq \emptyset\), then there is a basic feasible solution

如果有可行解,那么一定有BFS

ii) If there is an optimal solution for LP, there is a basic feasible solution that is optimal.

如果有最优解,那一定有是最优解的BFS

The simplex method

从一个BFS到一个BFS,让目标函数的值更优;

可以探测可行域是否空,可以探测解是否unbounded;

最关键的事情是,可以在不列举所有的BFS的情况下,向优化的方向移动;

基本的概念是,用合理的办法来对搜索下一个BFS时剪枝;

Reformulation of LP by the current BFS

Therefore, LP can be written as

\[\begin{array}{ll} \text { Minimize } & f=f_{0}-\sum_{j \in R}\left(z_{j}-c_{j}\right) x_{j} \\ \text { subject to } & \sum_{j \in R}\left(y_{j}\right) x_{j}+x_{B}=\bar{b} \\ & x_{j} \geq 0, j \in R, \text { and } x_{B} \geq 0 \end{array} \]

For this problem, \(x_{B}\) can be viewed as a slack variable, so the above LP can be further rewritten as

\[\begin{array}{ll} \text { Minimize } & f=f_{0}-\sum_{j \in R}\left(z_{j}-c_{j}\right) x_{j} \\ \text { subject to } & \sum_{j \in R}\left(y_{j}\right) x_{j} \leq \bar{b} \\ & x_{j} \geq 0, j \in R \end{array} \]

The simplex method in Tableau Format

\[\begin{array}{|c|c|c|c|} \hline & x_{B} & x_{N} & \text { RHS } \\ \hline f & 0 & c_{B} B^{-1} N-c_{N} & c_{B} B^{-1} b \\ \hline x_{B} & I & B^{-1} N & B^{-1} b \\ \hline \end{array} \]

Pivoting

posted @ 2022-10-31 16:37  miyasaka  阅读(73)  评论(0)    收藏  举报