BoydC5pt4

---

Previous part: BoydC5pt3

---

Examples

In this section we show by example that simple equivalent reformulations of a problem can lead to very different dual problems. We consider the following types of reformulations:

• introduce new variables and equality constraints
• make explicit constraints implicit or vice-versa
• transform objective or constraint functions

Introducing new variables and equality constraints

Ex 7.1.

\[\min~f(Ax+b) \]

reformulated problem and its dual

\[\begin{aligned} & \min & & f_0(y) \\ & s.t. & & Ax + b - y = 0 \end{aligned}~\overset{\text{dual}}{\Rightarrow}~\begin{aligned} & \max && b^T \nu - f_0^*(\nu) \\ & s.t. && A^T \nu = 0 \end{aligned} \]

Ex 7.2.

\[\min~\|Ax+b\| \]

reformulated problem and its dual

\[\begin{aligned} & \min & & \|y\| \\ & s.t. & & Ax + b - y = 0 \end{aligned}~\overset{\text{dual}}{\Rightarrow}~\begin{aligned} & \max && b^T \nu \\ & s.t. && A^T \nu = 0\\ & &&\|\nu\|_*\leq 1 \end{aligned} \]

holds for ex 3.1 in BoydC3.

Implicit constraints

Ex 7.3. LP with box constraints: primal and dual problem

\[\begin{aligned} & \min && c^T x \\ & s.t. && Ax = b \\ & && -\mathbf{1} \preceq x \preceq \mathbf{1} \end{aligned} \quad \quad \begin{aligned} & \max && -b^T\nu - \mathbf{1}^T\lambda_1 - \mathbf{1}^T\lambda_2 \\ & s.t. && c + A^T\nu + \lambda_1 - \lambda_2 = 0 \\ &&& \lambda_1 \succeq 0, \quad \lambda_2 \succeq 0 \end{aligned} \]

reformulation with box constraints made implicit

\[\begin{aligned} & \min \quad f_0(x) = \begin{cases} c^T x & -1 \leq x \leq 1 \\ \infty & \text{otherwise} \end{cases} \\ & s.t. \quad Ax = b \end{aligned} \]

its dual function

\[\begin{aligned} g(\nu) &= \inf_{-1 \preceq x \preceq 1} (c^T x + \nu^T (Ax - b)) \\ &= \sup_{\|x\|_{\infty}\leq 1} (-(c+A^T\nu)^T x)-b^T \nu\\ &= -b^T \nu - \|A^T \nu + c\|_1 \end{aligned} \]

dual problem: maximize \(g(\nu)= -b^T \nu - \|A^T \nu + c\|_1\)

Generalized inequalities

problem with generalized inequality constraints:

\[\begin{aligned} & \min && f_0(x) \\ & s.t. && f_i(x) \preceq_{K_i} 0, \quad i = 1, \ldots, m \\ &&& h_i(x) = 0, \quad i = 1, \ldots, p, \end{aligned} \]

where \(K_i\subset\mathbb{R}^{k_i}\) are proper cones.

The Lagrange dual

The associated Lagrangian is

\[L(x,\lambda,\nu) = f_0(x)+\sum_{k=1}^m\lambda_k^Tf_k(x)+\sum_{k=1}^p \nu_kh_k(x), \]

and the dual function is

\[g(\lambda,\nu) = \inf_{x\in\mathcal{D}} L(x,\lambda,\nu) \]

where \(\lambda_i\succeq_{K_i}0\), \(i\in\left[m\right]\).

Thm 9.1. Slater’s condition: strong duality \((d^* = p^*)\) holds when the primal problem is convex (\(f_0\) convex, \(f_i\) \(K_i\)-convex, \(h_i\) affine) and satisfies that \(\exists~x\in\mathbf{reint}~\mathcal{D}\) with \(Ax=b~(h_i(x)=0)\) and \(f_i(x)\prec 0\)

Optimality conditions

Thm 9.2. Complementary slackness: similar to thm 5.1 in BoydC5, assume strong duality holds

\[\lambda_i^{*T}f_i(x^*) = 0,\quad i=1,\ldots,m, \]

Thus

\[\lambda_i^* \succ_{K^*} 0 \implies f_i(x^*) = 0, \quad f_i(x^*) \prec_{K} 0 \implies \lambda_i^* = 0. \]

Thm 9.3. KKT conditions: similar to thm 5.2 in BoydC5

1. primal constraints: \(f_i(x^*)\preceq_{K_i} 0\), \(h_i(x^*)=0\)
2. dual constraints: \(\lambda^*_i\succeq_{K_i} 0\)
3. complementary slackness: \(\lambda_i^{*T} f_i(x^*)=0\)
4. gradient of Lagrangian w.r.t. \(x^*\) :

\[\nabla f_0(x^*) + \sum_{i=1}^{m} D f_i(x^*)^T\lambda_i^* + \sum_{i=1}^{p} \nu_i^* \nabla h_i(x^*) = 0 \]

where \(D f_i(x^*)=\frac{\partial f_i(x)}{\partial x}|_{x=x^*}\in\mathbb{R}^{k_i\times n}\).

• strong duality + primal optimal \(x^*\) and any dual optimal \((\lambda^*,\nu^*)\) \(\implies\) \((x^*,\lambda^*,\nu^*)\) satisfies KKT
• primal problem convex + strong duality + \((x^*,\lambda^*,\nu^*)\) satisfies KKT \(\implies\) primal optimal \(x^*\), dual optimal \((\lambda^*,\nu^*)\)

Perturbation and sensitivity analysis

Thm 9.4. similar to BoydC5, the inequality holds

\[p^*(u,v) \geq p^* - \sum_{i=1}^{m} \lambda_i^{*T} u_i - \nu^{*T} v, \]

and

\[\lambda_i^* = -\nabla_{u^*} p^*(0, 0) \]