BoydC2pt1
Notes for CVX101 and Convex Optimization(Boyd)-Chapter 2 Convex sets (part 1).
Affine and convex sets
Affine sets
Def 1.1. set \(C\) is affine \(\Leftrightarrow\) \(\forall~x_1,x_2 \in C\), \(\theta x_1 +(1-\theta) x_2\in C\), where \(\theta\in\mathbb{R}\).
Def 1.2. \(x\) is an affine combination of \(x_1,\cdots,x_k\) \(\Leftrightarrow\) \(x=\sum_{i=1}^k \theta_i x_i\), where \(\theta_i\in\mathbb{R}\) and \(\sum_{i=1}^k \theta_i=1\) .
An affine set contains every affine combination of its points.
Thm 1.1. If \(C\) is an affine set and \(x_0\in C\), then
is a subspace, \(i.e.\) closed under sums and scalar multiplication.
Cor. affine set \(C\) can be expressed as
\[C = V+x_0 := \{v+x_0~|~v\in V\} \]where \(V\) is a subspace, \(i.e.\) \(C\) can be expressed as a subspace plus an offset.
Def 1.3. affine hull of \(C\) :
The affine hull is the smallest affine set that contains \(C\)
Affine dimension and relative interior
Def 1.4. dimension of set \(C\) \(:=\) dimension of \(\mathbf{aff}~C\).
Def 1.5. the relative interior of the set \(C\)
where \(B(x,r)=\{y~|~\|y-x\| < r\}\).
Def 1.6. the relative boundary of set \(C\) \(:=\) \(\mathbf{cl}~C~\backslash~\mathbf{relint}~C\), where \(\mathbf{cl}~C\) is the
closure of \(C\).
Convex sets
Def 1.7. set \(C\) is convex \(\Leftrightarrow\) \(\forall~x_1,x_2 \in C\), \(\theta x_1 +(1-\theta) x_2\in C\), where \(\theta\in \left[0,1 \right]\).
The difference between convex set (def 1.7) and affine set (def 1.1) lies in the range of values that \(\theta\) can take.
Def 1.8. \(x\) is a convex combination of \(x_1,\cdots,x_k\) \(\Leftrightarrow\) \(x=\sum_{i=1}^k \theta_i x_i\), where \(\theta_i\geq 0\) and \(\sum_{i=1}^k \theta_i=1\) .
A set is convex if and only if it contains every convex combination of its points.
Def 1.9. convex hull of \(C\) :
The convex hull is the smallest convex set that contains \(C\).
Cones
Def 1.10. a set \(C\) is a cone (or nonnegative homogeneous) \(\Leftrightarrow\) \(\forall~x\in C\), \(\theta x\in C\), where \(\theta\geq 0\).
Def 1.11. a set \(C\) is a convex cone \(\Leftrightarrow\) \(\forall~x_1,x_2 \in C\), \(\theta_1 x_1 +\theta_2 x_2\in C\), where \(\theta_1,\theta_2\geq 0\).
Def 1.12. \(x\) is a conic combination (or a nonnegative linear combination) of \(x_1,\cdots,x_k\) \(\Leftrightarrow\) \(x=\sum_{i=1}^k \theta_i x_i\), where \(\theta_i \geq 0\) .
A convex cone contains every conic combination of its points.
Def 1.13. conic hull of \(C\) :
The conic hull is the smallest convex cone that contains \(C\).
Some important examples
Hyperplanes and halfspaces
Def 2.1. A hyperplane is a set of the form
where \(a\in\mathbb{R}^n\), \(a\neq\mathbf{0}\), \(b\in\mathbb{R}\).
It can also be represented as
\[\{x~|~a^T(x-x_0)=0\} = x_0 + a^{\perp} \]where \(a^Tx_0=b\), \(a^{\perp}=\{v~|~a^T v=0\}\).
Def 2.2. A hyperplane divides \(\mathbb{R}^n\) into two halfspaces. A (closed) halfspace is a set of the form
Halfspaces are convex, but not affine.
Halfspace can also be expressed as
\[\{x~|~a^T(x-x_0)\leq 0\} \]where \(a^Tx_0=b\).
Thm 2.1. The boundary of the halfspace is the hyperplane \(\{x~|~a^Tx=b\}\). The set \(\{x~|~a^T x < b\}\), which is the interior of the halfspace \(\{x~|~a^T x \leq b\}\), is called an open halfspace.
Euclidean balls and ellipsoids
Def 2.3. (Euclidean) ball with center \(x_c\) and radius \(r\):
Def 2.4. ellipsoid: set of the form
where \(P\in\mathbb{S}^n_{++}\), which can also be represented as
where \(A=P^{\frac{1}{2}}\). Ellipsoid is convex.
The matrix \(P\) determines how far the ellipsoid extends in every direction from \(x_c\)
Norm balls and norm cones
Def 2.5. norm ball with center \(x_c\) and radius \(r\): \(\{x~|~\|x-x_c\|\leq r\}\) (is convex).
Def 2.6. norm cone: \(\{(x,t)~|~\|x\|\leq t\}\) (is convex).
Polyhedra
Def 2.7. polyhedra: solution set of finitely many linear inequalities and equalities
where \(A\in \mathbb{R}^{m\times n}\), \(C\in\mathbb{R}^{p\times n}\).
polyhedron is intersection of finite number of halfspaces and hyperplanes, so it's convex.
Positive semidefinite cone
Thm 2.2. The set \(\mathbb{S}_{+}^n\) is a convex cone.
Operations that preserve convexity (CPO)
Intersection
Thm 3.1. \(S_1\) and \(S_2\) are convex \(\Rightarrow\) \(S_1\cap S_2\) is convex.
This property extends to the intersection of an infinite number of sets.
Ex 3.1.
\[S=\{x\in\mathbb{R}^m~|~\left| p(t) \right|\leq 1~\text{for}~\left|t\right|\leq\frac{\pi}{3}\} \]where \(p(t)=\sum_{k=1}^m x_k \cos(kt)\) \(\Rightarrow\) \(S\) is convex.
proof. let \(S_t = \{x\in\mathbb{R}^m~|~ -1\leq \left[\cos t,\cos 2t,\cdots \cos mt \right]^T x \leq 1\}\), then \(S_t\) is convex \(\Rightarrow\) \(S=\cap_{\left|t\right|\leq\frac{\pi}{3}} S_t\) is convex.
Ex 3.2. The positive semidefinite cone \(\mathbb{S}_n^+\) is convex because it can be expressed as
\[\bigcap_{z \neq \mathbf{0}} \{X\in \mathbb{S}^n~|~z^T Xz\geq 0\} \]
Affine functions
Rec. \(f:\mathbb{R}^n\to\mathbb{R}^m\) is affine \(\Leftrightarrow\) \(f\) is a sum of a linear function and a constant \(i.e.\) \(f(x)=Ax+b\).
Thm 3.2. \(S\) is convex \(\Leftrightarrow\) \(f(S)=\{f(x)~|~x\in S\}\) (the image of \(S\) under \(f\)) is convex, where \(f\) is affine.
proof. \(S\) is convex \(\Leftrightarrow\) \(\forall~x_1,x_2\in S\), \(\theta x_1+(1-\theta) x_2 \in S\) \(\Leftrightarrow\) \(\forall~f(x_1),f(x_2)\in f(S)\), \(f(\theta x_1+(1-\theta) x_2) \in f(S)\) \(\Leftrightarrow\) \(\forall~f(x_1),f(x_2)\in f(S)\), \(\theta f(x_1)+(1-\theta)f(x_2) \in f(S)\) \(\Leftrightarrow\) \(f(S)\) is convex.
Thm 3.3. If \(f:\mathbb{R}^n\to\mathbb{R}^m\) is affine, the inverse image of \(S\) under \(f\),
is convex, where \(S\) is convex.
proof. \(\forall~x\in f^{-1}(S)\), \(f(x)\in S\) \(\Rightarrow\) \(f(f^{-1}(S))\subseteq S\). Thus \(f(f^{-1}(S))\subseteq S\cap f(\mathbb{R}^n)\). \(\forall~y\in S\cap f(\mathbb{R}^n)\), \(\exists~x\in \mathbb{R}^n\) \(s.t.\) \(y=f(x)\), therefore \(f(x)\in S\), which means this \(x\in f^{-1}(S)\) and \(y=f(x)\in f(f^{-1}(S))\). Thus, \(S\cap f(\mathbb{R}^n)\subseteq f(f^{-1}(S))\). Therefore \(f(f^{-1}(S))=S\cap f(\mathbb{R}^n)\). \(f(\mathbb{R}^n)\) is convex clearly, so \(f(f^{-1}(S))\) is convex (thm 3.1). Through thm 3.2, \(f^{-1}(S)\) is convex.
Rmk. Since \(f:\mathbb{R}^n\to C\) is not necessarily surjective, it is not guaranteed that \(f(f^{-1}(C))=C\).
Thm 3.4. The projection of a convex set onto some of its coordinates is convex: if \(S\in\mathbb{R}^m\times\mathbb{R}^n\), then
is convex.
Thm 3.5. If \(S_1\) and \(S_2\) are convex, then
- \(S_1+S_2 = \{x_1+x_2~|~x_1\in S_1, x_2\in S_2\}\) is convex.
- \(S_1\times S_2 = \{(x_1,x_2)~|~x_1\in S_1, x_2\in S_2\}\) is convex
The image of this set under the linear function \(f(x_1, x_2) = x_1 + x_2\) is the sum \(S_1 + S_2\).
- the partial sum \(S=\{(x,y_1+y_2)~|~(x,y_1)\in S_1, (x,y_2)\in S_2\}\) is convex.
Ex 3.3. Solution set of linear matrix inequality(LMI). Let \(A(x)=\sum_{k=1}^n x_k A_k\), then \(S:=\{x~|~A(x)\preceq B\}\) is convex, where \(A_i,B\in\mathbb{S}^m\).
proof. let \(f(x)=B-A(x)\), then \(f:\mathbb{R}^n\to\mathbb{S}^m\) is affine and \(S\) is the inverse image of \(\mathbb{S}_m^+\) under \(f\). Thus \(S\) is convex.
Ex 3.4. Hyperbolic cone. The set
\[S:=\{x~|~x^T P x\leq (c^Tx)^2, c^Tx\geq 0\} \]is convex, where \(P\in\mathbb{S}_n^+\), \(c\in\mathbb{R}^n\).
proof. Let \(f(x)=(P^{\frac{1}{2}}x,c^T x)\), \(S\) can be seen as the inverse image of set \(S^\prime\) under \(f\), where
\[S^\prime := \{ (z,t)~|~z^Tz\leq t^2, t\geq 0 \} \]
Linear-fractional and perspective functions
Def 3.1. \(P:\mathbb{R}^{n+1}\to\mathbb{R}^n\) is a perspective function \(\Leftrightarrow\) \(P(z,t)=\frac{z}{t}\), where \(\mathbf{dom}~P=\mathbb{R}^n\times\mathbb{R}_{++}\).
Thm 3.6. For perspective function \(P\), if \(C\subseteq\mathbf{dom}~P\) is convex, \(P(C)\) is convex.
proof. If \(C\) is convex, \(\forall~\frac{z_1}{t_1},\frac{z_2}{t_2}\in P(C)\), then
\[\theta \frac{z_1}{t_1} + (1-\theta) \frac{z_2}{t_2} = \frac{\tilde{\theta}z_1 + (1-\tilde{\theta})z_2}{\tilde{\theta}t_1 + (1-\tilde{\theta})t_2}\in P(C) \]where \(\tilde{\theta} = \frac{\theta t_2}{\theta t_2+(1-\theta)t_1}\). So \(P(C)\) is convex.
Rmk. Since the value of \(~\tilde{\theta}~\) is independent of \(z_1\) and \(z_2\), we can set \(z_1=\mathbf{0}\) (or \(z_2=\mathbf{0}\)) to quickly solve for \(~\tilde{\theta}~\).
Thm 3.7. For perspective function \(P:\mathbb{R}^{n+1}\to\mathbb{R}^{n}\), if \(C\subseteq\mathbb{R}^{n}\) is convex, \(P^{-1}(C)\) is convex.
Def 3.2. Suppose \(g:\mathbb{R}^n\to\mathbb{R}^{n+1}\) is affine, \(i.e.\)
\(P\) is a perspective function, \(f=P\circ g\) \(i.e.\)
is a linear-fractional (or projective) function.
If \(c=\mathbf{0}\) and \(d>0\), \(f\) is an affine function. So affine and linear functions are special cases of linear-fractional functions.
Thm 3.8. If \(f\) is a linear-fractional function, \(C\) is convex and \(C\subseteq \mathbf{domain}~f\), then \(f(C)\) is convex.
Thm 3.9. If \(C\subseteq\mathbb{R}^m\) is convex, \(f^{-1}(C)\) is convex, where \(f:\mathbb{R}^n\to\mathbb{R}^m\) is a linear-fractional function.
Ex 3.5. Conditional probabilities. Let \(p_{i,j}\) denote \(\mathbf{prob}(u=i,v=j)\), then the conditional probability \(f_{i,j}=\mathbf{prob}(u=i~|~v=j)\) is
\[f_{i,j} = \frac{p_{i,j}}{\sum_{k=1}^n p_{k,j}} \]Thus \(f\) is obtained by a linear-fractional mapping from \(p\).
Next part: BoydC2pt2
浙公网安备 33010602011771号