# 2 Main Layout Conventions of Matrix Calculus

## Numerator Layout

Vector by vector 符合直观. Jacobian.

$\frac{\partial y}{\partial x} = \begin{pmatrix} \frac{\partial y_1}{\partial x_1} & \dots & \frac{\partial y_1}{\partial x_n}\\ \vdots & \ddots &\vdots\\ \frac{\partial y_m}{\partial x_1} & \dots & \frac{\partial y_m}{\partial x_n} \end{pmatrix}$

Scalar by matrix 要做一次转置, 不舒服.

$\frac{\partial z}{\partial A}= \begin{pmatrix} \frac{\partial z}{\partial a_{11}} & \dots & \frac{\partial z}{\partial a_{m1}}\\ \vdots & \ddots &\vdots\\ \frac{\partial z}{\partial a_{1n}} & \dots & \frac{\partial z}{\partial a_{mn}} \end{pmatrix}$

Chain rule 符合直观.

$\frac{\partial f\circ g}{\partial x} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial x}$

## Denominator Layout

Vector by vector 不舒服. Hessian.

$\frac{\partial y}{\partial x} = \begin{pmatrix} \frac{\partial y_1}{\partial x_1} & \dots & \frac{\partial y_m}{\partial x_1}\\ \vdots & \ddots &\vdots\\ \frac{\partial y_1}{\partial x_n} & \dots & \frac{\partial y_m}{\partial x_n} \end{pmatrix}$

Scalar by matrix 舒服.

$\frac{\partial z}{\partial A}= \begin{pmatrix} \frac{\partial z}{\partial a_{11}} & \dots & \frac{\partial z}{\partial a_{1n}}\\ \vdots & \ddots &\vdots\\ \frac{\partial z}{\partial a_{m1}} & \dots & \frac{\partial z}{\partial a_{mn}} \end{pmatrix}$

Chain rule "倒过来" 了, 不舒服.

$\frac{\partial f\circ g}{\partial x} = \frac{\partial g}{\partial x}\frac{\partial f}{\partial g}$

## 混用

posted @ 2019-08-30 15:53  resolvent  阅读(186)  评论(0编辑  收藏  举报