Kronecker Products and Stack Operator

目录

• 定义
  • Stack Operator
  • Kronecker Product
• 性质
  • Stack Operator
  • Kronecker Product
    • \(a \otimes b = vec(b a^T)\)
    • \((A \otimes B)^T = (A^T \otimes B^T)\)
    • 半线性
    • \((A \otimes B) (C\otimes D) = (AC \otimes BD)\)
    • \((A \otimes B)^{-1} = (A^{-1} \otimes B^{-1})\)
    • \(\mathrm{det}(A_{n\times n} \otimes B_{m \times m}) = \mathrm{det}(A)^m \cdot \mathrm{det}(B)^n\)
    • \(\mathrm{Tr}(A \otimes B) = \mathrm{Tr}(A) \cdot \mathrm{Tr}(B)\)
    • \(vec(ABC) = (C^T \otimes A) vec(B)\)

Whitcomb L. Notes on Kronecker Products.

定义

Stack Operator

对于任意的矩阵\(A \in \mathbb{R}^{m \times n}\),

\[vec(A) := [A_{00}, A_{10}, \ldots, A_{m-1,n-1}]^T \in \mathbb{R}^{mn}, \]

即按列展开.

Kronecker Product

对于任意的矩阵\(A \in \mathbb{R}^{m\times n }, B \in \mathbb{R}^{p \times q}\),

\[A \otimes B := \left [ \begin{array}{ccc} A_{00} \cdot B & \cdots & A_{0n-1} \cdot B \\ \vdots & \ddots & \vdots \\ A_{m-1,0} \cdot B & \cdots & A_{m-1,n-1} \cdot B \end{array} \right ] \in \mathbb{R}^{mp \times nq} \]

性质

Stack Operator

\[\mathrm{Tr}(A^TB) = vec(A)^T vec(B). \]

Kronecker Product

易知,

\[[A \otimes B]_{ip+s,jq+t} = A_{i,j} \cdot B_{s,t}, \quad i\in [m], s\in[p], j \in [n], t \in [q], \]

这里\([m] = \{0, 1, \ldots, m-1\}\).

\(a \otimes b = vec(b a^T)\)

• \(a \in \mathbb{R}^m, b \in \mathbb{R}^n\), 则
  \[a \otimes b = vec(b a^T) \]

\((A \otimes B)^T = (A^T \otimes B^T)\)

\((A \otimes B)^T = (A^T \otimes B^T)\)是显然的.

故

\[[A \otimes B]_{*, jq+t} = A_{*,j} \otimes B_{*, t} = vec(B_{*, t} A_{*,j}^T) \\ [A \otimes B]_{ip+s, *}^T = A_{i,*}^T \otimes B_{s,*}^T = vec(B^T_{s,*} A_{i,*}). \]

半线性

• \(A \otimes \alpha B = \alpha A \otimes B = \alpha (A \otimes B).\)

• \[(A+B) \otimes C = A \otimes C +B \otimes C \\ A \otimes (B+C) = A\otimes B + A \otimes C. \]

• \((A \otimes B) \otimes C=A \otimes (B\otimes C)\):

  \[\begin{array}{ll} (A \otimes B) \otimes C &= [A_{i,j} \cdot B_{s,t} \cdot C ]\\ &= A \otimes (B \otimes C). \end{array} \]

• 通常 \((A \otimes B) \not= (B \otimes A)\).

\((A \otimes B) (C\otimes D) = (AC \otimes BD)\)

\[\begin{array}{ll} [(A \otimes B) (C\otimes D)]_{ip+s, jq+t} &= [A \otimes B]_{ip+s, *} [C\otimes D]_{*,jq+t} \\ &= vec(B_{s, *}^TA_{i,*})^T vec(D_{*,t} C_{*,j}^T) \\ &= \mathrm{Tr}(A_{i,*}^TB_{s,*}D_{*,t} C_{*,j}^T) \\ &= \mathrm{Tr}(C_{*,j}^TA_{i,*}^TB_{s,*}D_{*,t}) \\ &= A_{i, *}C_{*,j} \cdot B_{s, *} D_{*,t} \\ &= [AC]_{ij} \cdot [BD]_{st} \\ &= [AC \otimes BD]_{ip+s,jq+t}. \end{array} \]

\((A \otimes B)^{-1} = (A^{-1} \otimes B^{-1})\)

条件自然是A, B为满秩方阵:

\[(A \otimes B) (A^{-1} \otimes B^{-1}) = (AA^{-1} \otimes BB^{-1}) = I \]

\(\mathrm{det}(A_{n\times n} \otimes B_{m \times m}) = \mathrm{det}(A)^m \cdot \mathrm{det}(B)^n\)

就像用普通的高斯消去法将矩阵化为对角型一样, 在对\(A_{n\times n } \otimes B_{m\times m}\)消去的过程中可以发现, \(B\)不会产生丝毫的影响, 结果便是显而易见的了.

\(\mathrm{Tr}(A \otimes B) = \mathrm{Tr}(A) \cdot \mathrm{Tr}(B)\)

\[\mathrm{Tr}(A \otimes B) = \sum_{i=1}^m \sum_{j=1}^n A_iB_j = \mathrm{Tr}(A) \cdot \mathrm{Tr}(B). \]

\(vec(ABC) = (C^T \otimes A) vec(B)\)

设\(A \in \mathbb{R}^{m\times n}, B \in \mathbb{R}^{n \times p}, C \in \mathbb{R}^{p \times q}\),

\[[vec(ABC)]_{jm+i} = [ABC]_{i,j} = \mathrm{Tr}(A_{i,*}BC_{*,j}) = \mathrm{Tr}(C_{*,j}A_{i,*}B)=vec(A_{i,*}^TC_{*j}^T)^T vec(B) = [C^T \otimes A]_{jm+i,*} vec(B) \]

特例:

\[Ax = IAx = vec(IAx) = (x^T \otimes I)vec(A) \]

这个在处理梯度的时候会比较有用:

\[y = Ax \]

则

\[\mathrm{d}y = (\mathrm{d}A)x + A\mathrm{d}x = (x^T \otimes I) vec(\mathrm{d}A) + A \mathrm{d}x. \]