BoydC2pt2
Notes for CVX101 and Convex Optimization(Boyd)-Chapter 2 Convex sets (part 2).
Previous part: BoydC2pt1
Generalized inequalities
Proper cones and generalized inequalities
Def 4.1. a convex cone \(K\subseteq\mathbb{R}^n\) is a proper cone if
- \(K\) is closed (contains its boundary)
- \(K\) is solid (has nonempty interior)
- \(K\) is pointed (contains no line \(i.e.\) \(x\in K\), \(-x\in K\) \(\Rightarrow\) \(x=0\))
Ex 4.1 some examples of proper cones:
- nonnegative orthant \(K=\mathbb{R}_+^n\)
- positive semidefinite cone \(K=\mathbb{S}_+^n\)
- nonnegative polynomials on \(\left[ 0,1 \right]\)
\[K=\{x\in\mathbb{R}^n~|~x_1+\sum_{k=1}^{n-1} x_{k+1} t^{k}\geq 0~\text{for}~ t\in\left[ 0,1 \right]\} \]
Def 4.2. generalized inequality defined by a proper cone \(K\):
Rmk. \(\preceq_K\) is not in general a linear ordering: we can have \(x\not\preceq y\) and \(y\not\preceq x\).
Minimum and minimal elements
Def 4.3. \(x\in S\) is the minimum element of \(S\) with respect to \(\preceq_K\) if
Def 4.4. \(x\in S\) is the minimal element of \(S\) with respect to \(\preceq_K\) if
Ex 4.2. This example shows the difference between the minimum element and the minimal element:
\(x_1\) is the minimum element of \(S_1\), \(x_2\) is the minimal element of \(S_2\).
Separating and supporting hyperplanes
Separating hyperplane theorem
Thm 5.1. separating hyperplane theorem: If \(C\) and \(D\) are nonempty disjoint convex sets, there exists \(a\neq\mathbf{0}\) \(s.t.\)
\(i.e.\) the hyperplane \(\{ x~|~a^Tx=b \}\) separates \(C\) and \(D\).
strict separation (\(a^Tx<b\) for \(x\in C\), \(a^Tx>b\) for \(x\in D\)) requires additional assumptions (\(e.g.\) \(C\) is closed and \(D\) is a singleton).
Rmk. The converse of the separating hyperplane theorem (\(i.e.\) existence of a separating hyperplane implies that C and D do not intersect) is not true.
Thm 5.2. Any two convex sets \(C\) and \(D\), at least one of which is open, are disjoint if and only if there exists a separating hyperplane.
Supporting hyperplane theorem
Def 5.1. supporting hyperplane to set \(C\) at boundary point \(x_0\) (\(i.e.\) \(x_0\in\mathbf{bd}~C\)):
where \(a\neq\mathbf{0}\) and \(a^Tx\leq a^T x_0\) for \(x\in C\).
Thm 5.3. supporting hyperplane theorem: if \(C\) is convex, then there exists a supporting hyperplane at every boundary point of \(C\).
Thm 5.4. If a set is closed, has nonempty interior, and has a supporting hyperplane at every point in its boundary, then it is convex.
It's a partial converse of the supporting hyperplane theorem (thm 5.3).
Dual cones and generalized inequalities
Dual cones
Def 6.1. Dual cone of a cone \(K\): \(K^*=\{y~|~\forall~x\in K,~y^Tx\geq 0\}\)
Dual cones satsisfy several properties, such as:
- \(K^*\) is closed and convex.
- \(K_1\subseteq K_2\) \(\Rightarrow\) \(K_2^*\subseteq K_1^*\).
- \(\mathbf{int}~K\neq \emptyset\) \(\Rightarrow\) \(K^*\) is pointed.
- \(\mathbf{cl}~K\) is pointed \(\Rightarrow\) \(\mathbf{int}~K^*\neq \emptyset\)
- \(K^{**}=\mathbf{conv}~K\). (Hence if \(K\) is convex and closed, \(K^{**} = K\))
Property 1 is evidently valid (\(K^*\) can be regarded as the intersection of halfspaces).
Property 2 is also evidently valid: \(K_1^*\) \(=\) \(\cap_{x\in K_1}\{y~|~x^Ty\geq 0\}\), \(K_2^*\) \(=\) \(\cap_{x\in K_2}\{y~|~x^Ty\geq 0\}\) \(=\) \(\cap_{x\in K_1}\{y~|~x^Ty\geq 0\}\) \(\cap\) \(\cap_{x\in K_2\backslash K_1}\{y~|~x^Ty\geq 0\}\) \(=\) \(K_1^*\cap \cap_{x\in K_2\backslash K_1}\{y~|~x^Ty\geq 0\}\).
For \(y,-y\in K^*\), \(\forall~x\in \mathbf{int}~K\), \(\exists~\varepsilon_x>0\), \(B(x,\varepsilon_x)\subseteq \mathbf{int}~K\). \(y^Tx\geq 0\) and \(-y^Tx\geq 0\) \(\Rightarrow\) \(y^Tx=0\). Thus, \(\forall~x\in \mathbf{int}~K\), \(y^Tx=0\), furthermore if \(\|y\|_2> 0\), \(x+\frac{\varepsilon_x}{2\|y\|_2}y\) \(\in\) \(B(x,\varepsilon_x)\subseteq \mathbf{int}~K\), but \(y^T(x+\frac{\varepsilon_x}{2\|y\|_2}y)\) \(=\) \(\frac{\varepsilon_x}{2}\|y\|_2>0\), which contradicts \(\forall~x\in \mathbf{int}~K\), \(y^Tx=0\). Thus \(\|y\|_2= \mathbf{0}\) \(i.e.\) \(y=0\). Property 3 is valid.
Thm 6.1. If \(K\) is a proper cone, then so is its dual \(K^∗\), and moreover, that \(K^{**} = K\).
Dual generalized inequalities
- \(x\preceq_K y\) \(\Leftrightarrow\) \(\forall ~\lambda\succeq_{K^*}\mathbf{0}\), \(\lambda^T x\leq \lambda^T y\).
- \(x\prec_K y\) \(\Leftrightarrow\) \(\forall ~\lambda\succeq_{K^*}\mathbf{0},~\lambda\neq\mathbf{0}\), \(\lambda^T x< \lambda^T y\).
Minimum and minimal elements via dual inequalities
Thm 6.2. \(x\) is minimum element of \(S\) \(w.r.t.\) \(\preceq_K\) iff for all \(\lambda\succ_{K^*} \mathbf{0}\), \(x\) is the unique minimizer of \(\lambda^Tz\) over \(S\).
Thm 6.3. If \(x\) minimizes \(\lambda^T z\) over \(S\) for some \(\lambda\succ_{K^*}\mathbf{0}\) then \(x\) is minimal.
Thm 6.4. If \(x\) is a minimal element of a convex set \(S\), then there exists a nonzero \(\lambda\succeq_{K^*}\mathbf{0}\) \(s.t.\) minimizes \(\lambda^T z\) over \(S\).
Next chapter: BoydC3pt1
浙公网安备 33010602011771号