Orthogonal Complement

目录

• Orthogonal Complement in Linear Algebra
• Orthogonal Complement in Functional Analysis
• Problem-Solving

Orthogonal Complement in Linear Algebra

Definition. Let \(V\) be an inner product space and \(S\) a nonempty subset of \(V\). The orthogonal complement of \(S\) is

\[S^{\perp}:=\{x\in V:\langle x,y\rangle=0,\forall y\in S\} \]

Example. For any matrix \(A\in M_{m\times n}(\mathbb{C})\), \(\text{im}(L_{A^*})^{\perp}=\ker(L_A)\).

Theorem. Let \(W\) be a finite-dimensional subspace of an inner product space \(V\). Then for any \(y\in V\), there exists a unique pair \((u,z)\in W\times W^{\perp}\) such that \(y=u+z\) . It follows that \(V=W\oplus W^{\perp}\), and hence if \(\dim V<\infty\), then \(\dim V=\dim W+\dim W^{\perp}\). Moreover, if \(\{v_1,v_2,\cdots,v_k\}\) is an orthonormal basis for \(W\), then

\[u=\sum_{i=1}^{k}\langle y,v_i\rangle v_i \]

Here \(u\) is called the orthogonal projection of \(y\) on \(W\), and in \(W\) it is the closest vector to \(y\), i.e.,

\[\left\|y-u\right\|\le\left\|y-x\right\|,\quad\forall x\in W \]

Remark. By Replacement Theorem and Gram-Schmidt process, an orthogonal set \(S=\{v_1,v_2,\cdots,v_k\}\) in an \(n\)-dimensional inner product space \(V\) can be extended to an orthonormal basis \(\{v_1,v_2,\cdots,v_k,v_{k+1},\cdots,v_n\}\) for \(V\). Denote \(W=\text{span}(S)\), then \(S'=\{v_{k+1},\cdots,v_n\}\) is an orthonormal basis for \(W^{\perp}\).

Proposition. (1) Let \(W_1,W_2\) be subspaces of an inner product space \(V\). Then

\[(W_1+W_2)^{\perp}=W_1^{\perp}\cap W_2^{\perp} \]

Further, if \(V\) is finite-dimensional, then (apply (3) below)

\[(W_1\cap W_2)^{\perp}=W_1^{\perp}+W_2^{\perp} \]

(2) Let \(S,S_0\) be subsets of inner product space \(V\). Then

\[S_0\subset S\Rightarrow S_0^{\perp}\supset S^{\perp}\\ S\subset (S^{\perp})^{\perp},\ \text{span}(S)\subset(S^{\perp})^{\perp} \]

(3) Let \(W\) be a finite-dimensional subspace of an inner product space \(V\). Then for any \(x\notin W\), there exists \(y\in W^{\perp}\) such that \(\langle x,y\rangle\neq0\). Consequently, \((W^{\perp})^{\perp}=W\).

Without the assumption \(\dim W<\infty\) , the statement (3) is false.

Counterexample. Let \(F\) be any field. A sequence in \(F\) is a function

\[\sigma:\mathbb{N}^+\to F\quad\sigma(n)=a_n,\forall n\in \mathbb{N}^+ \]

and is denoted by \(\{a_n\}\). Let \(V\) consist of all sequences \(\{a_n\}\) in \(F\) that have only a finite number of nonzero terms \(a_n\). If \(\{a_n\}\) and \(\{b_n\}\) are in \(V\) and \(t\in F\), define

\[\{a_n\} + \{b_n\} = \{a_n + b_n\},\ t\{a_n\} = \{ta_n\} \]

With these operations \(V\) is a vector space.

Now let \(F=\mathbb{R}\) or \(\mathbb{C}\).

• Fisrt we define an inner product on \(V\) by

\[\langle \sigma,\mu \rangle=\sum_{n=1}^{\infty}\sigma(n)\overline{\mu(n)},\quad \forall \sigma,\mu\in V \]

Note that since all but a finite number of terms of the series are \(0\), the series converges.

• Then we construct an orthonormal basis for \(V\).

Let \(e_n\) be a sequence s.t. \(e_n(k)=\delta_{n,k}\), where \(\delta_{n,k}\) is the Kronecker delta.

Clearly \(\{e_1,e_2,\cdots\}\) is an orthonormal basis for \(V\).

• Finally we construct an infinite-dimensional subspace of \(V\) that fails the statement.

Let \(\sigma(n)=e_1+e_n\) and \(W=\text{span}(\{\sigma_n:n\ge 2\})\).

Clearly \(e_1\notin W\), so \(W\neq V\). But \(W^{\perp}=\{0\}\) (as showed below), thus \((W^{\perp})^{\perp}\neq W\).

\[\boxed{ \begin{array}\\ x\in W^{\perp}&&(c_1,c_2,c_3,\cdots,c_N,0,0,\cdots)&&c_1+c_2=0\\ \sigma_2&&(1,1,0,\cdots,0,0,0,\cdots)&&c_1+c_2=0\\ \sigma_3&&(1,0,1,\cdots,0,0,0,\cdots)&&c_1+c_3=0\\ \ \vdots&&\qquad\qquad\vdots&&\qquad\vdots\\ \sigma_{N}&&(1,0,0,\cdots,1,0,0,\cdots)&&c_1+c_{N}=0\\ \sigma_{N+1}&&(1,0,0,\cdots,0,1,0,\cdots)&&c_1+0=0\\ \ \vdots&&\qquad\qquad\vdots&&\qquad\vdots\\ \end{array}\\ \ \\ \ \\ \Rightarrow c_1=c_2=\cdots=c_N=0\quad\Rightarrow x=0\qquad\qquad\qquad\qquad } \]

Remark. The statement (3) will hold whenever the orthogonal projection map of \(V\) on \(W\) exists, i.e., each vector in \(V\) has an orthogonal projection in \(W\). (https://math.stackexchange.com/questions/3567361/question-regarding-orthogonal-complement)

Orthogonal Complement in Functional Analysis

Problem-Solving

Problem. Let \(L\subset \mathbb{R}^{n+l}\ (n,l\in \mathbb{Z}_{\ge 1})\) be an \(n\)-dimensional subspace and let \(\{v_1,\cdots,v_{n+l}\}\) be a basis of \(\mathbb{R}^{n+l}\). Show that there exists \(1\le j_1<\cdots<j_n\le n+l\) such that the restriction \(P_{j_1,\cdots,j_n}|_L:L\to \mathbb{R}^{n+l}\) is injective (and hence induces an isomphism), where \(P_{j_1,\cdots,j_n}\) is the orthogonal projection of \(\mathbb{R}^{n+l}\) on \(\text{span}\{v_{j_1},\cdots,v_{j_n}\}\).

Proof I (Haosen). The orthogonal complement of the kernel of \(P_{j_1,\cdots,j_n}|_L\) is

\[\begin{align*} \ker(P_{j_1,\cdots,j_n}|_L)^{\perp} &=\left(\ker(P_{j_1,\cdots,j_n})\cap L\right)^{\perp}\\ &=\left(\text{span}(\{v_{j_1},\cdots,v_{j_n}\})^{\perp}\cap L\right)^{\perp}\\ &=\text{span}(\{v_{j_1},\cdots,v_{j_n}\})+L^{\perp} \end{align*}\]

Therefore \(P_{j_1,\cdots,j_n}|_L\) is injective iff \(\text{span}(\{v_{j_1},\cdots,v_{j_n}\})+L^{\perp}=\mathbb{R}^{n+l}\).

Let \(\{w_1,\cdots,w_l\}\) be a basis of the \(l\)-dimensional subspace \(L^{\perp}\). There exists \(A\in M_{(n+l)\times l}(\mathbb{R})\) of rank \(l\) such that

\[(w_1\ \cdots\ w_l)=(v_1\ \cdots\ v_{n+l})A \]

Now we note that there exists a sequence of elementary column operations that transforms \(A\) into a matrix with exactly \(n\) zero rows, in other words, there exists \(Q\in GL_{l}(\mathbb{R})\) such that \(AQ\) has exactly \(n\) zero rows, with the other \(l\) nonzero rows being linearly independent. We claim that the labels of the zero rows, denoted by \(1\le j_1<\cdots<j_n\le n+l\), will fulfill the proof. For clarity, let \(1\le i_1<\cdots<i_l\le n+l\) denote the labels of the nonzero rows.

In fact, \(L^{\perp}=\text{span}(\{w_1,\cdots,w_l\})=\) the column space of the matrix \((w_1\ \cdots\ w_l)=\) the column space of the matrix \((w_1\ \cdots\ w_l)Q=\) the column space of the matrix \((v_1\ \cdots\ v_{n+l})AQ=\text{span}(\{v_{i_1},\cdots,v_{i_l}\})\). Hence, \(\text{span}(\{v_{j_1},\cdots,v_{j_n}\})+L^{\perp}=\text{span}(\{v_1,\cdots,v_{n+l}\})=\mathbb{R}^{n+l}\). \(\blacksquare\)

Proof II (Shihe). This proof is geometrically intuitive. For each \(1\le k\le n+l\), take a nonzero vector \(w_k\in \mathbb{R}^{n+l}\) such that \(\text{span}(\{w_k\})=\text{span}(\{v_1,\cdots,v_{n+l}\}\setminus\{v_k\})^{\perp}\). We claim that \(w_1,\cdots,w_{n+l}\) are linearly independent.

In fact, if \(\sum_{k=1}^{n+l}\lambda_k w_k=0\) (\(\lambda_1,\cdots,\lambda_{n+l}\in \mathbb{R}\)), then by applying \(\langle \cdot,v_k\rangle\) to both sides we get \(\lambda_k\langle w_k,v_k\rangle=0\) (\(k=1,\cdots,n+l\)). Clearly \(\langle w_k,v_k\rangle\neq 0\), because otherwise \(w_k\) is orthogonal to every vector in the basis \(\{v_1,\cdots,v_{n+l}\}\) and hence must be zero, contradiction. Thus \(\lambda_k=0\ (k=1,\cdots,n+l)\) and hence \(w_1,\cdots,w_{n+l}\) are linearly independent.

Since \(\dim L=n<n+l\), there exists some \(w_{k_1}\) that is not in \(L\). The orthogonal projection of \(\mathbb{R}^{n+l}\) along \(\text{span}(\{w_{k_1}\})\), denoted by \(P_{w_{k_1}}:\mathbb{R}^{n+l}\to \mathbb{R}^{n+l}\), induces an isomorphism when restricted to \(L\), as \(\ker(P_{w_{k_1}}|_L)=\text{span}(\{w_{k_1}\})\cap L=0\). Moreover,

\[\text{im}(P_{w_{k_1}})=\ker(P_{w_{k_1}})^{\perp}=\text{span}(\{w_{k_1}\})^{\perp}=\text{span}(\{v_1,\cdots,v_{n+l}\}\setminus\{v_{k_1}\}) \]

If \(l=1\), we're done. If \(l\ge 2\), then \(\dim P_{w_{k_1}}(L)=n<n+l-1\) and so there exists \(k_2\neq k_1\) such that \(w_{k_2}\notin P_{w_{k_1}}(L)\). The orthogonal projection of \(\text{im}(P_{k_1})\) along \(\text{span}(\{w_{k_2}\})\), denoted by \(P_{w_2}:\text{im}(P_{w_1})\to \text{im}(P_{w_1})\), induces an isomorphism when restricted to \(P_{w_{k_1}}(L)\), as \(\ker(P_{w_{k_2}}|_{P_{w_{k_1}}(L)})=\text{span}(\{w_{k_2}\})\cap P_{w_1}(L)=0\). Moreover,

\[\text{im}(P_{w_{k_2}})=\ker(P_{w_{k_2}})^{\perp}=\text{span}(\{w_{k_2}\})^{\perp}=\text{span}(\{v_1,\cdots,v_{n+l}\}\setminus\{v_{k_1},v_{k_2}\}) \]

Continue this process inductively until we get \(P_{w_l}\). Then

\[P:\mathbb{R}^{n+l}\to \mathbb{R}^{n+l}\quad P(v):=P_{w_l}\circ\cdots\circ P_{w_2}\circ P_{w_1}(v) \]

is a well-defined orthogonal projection of \(\mathbb{R}^{n+l}\) on \(\text{span}(\{v_1,\cdots,v_{n+l}\}\setminus\{v_{k_1},\cdots,v_{k_l}\})\), fulfilling the proof. \(\blacksquare\)