# 矩阵求导

$\dfrac{dy}{dx} = \dfrac{df(x)}{dx}, dy = \dfrac{df(x)}{dx}dx,(1)$

$\dfrac{\partial y}{\partial x_{1}} = \dfrac{\partial f(x)}{dx_{1}}, \dfrac{\partial y}{\partial x_{2}} = \dfrac{\partial f(x)}{dx_{2}}, dy = \dfrac{\partial f(x)}{dx_{1}} dx_{1} + \dfrac{\partial f(x)}{dx_{2}} dx_{2},(2)$

$dy = \dfrac{\partial f(x)}{dx_{1}} dx_{1}+\dfrac{\partial f(x)}{dx_{2}} dx_{2}+...+\dfrac{\partial f(x)}{dx_{n}} dx_{n} = \sum_{i=1}^{n} \dfrac{\partial f(x)}{dx_{i}} dx_{i},(3)$

$\dfrac{\partial f(x)}{dx} = (\dfrac{\partial f(x)}{dx_{1}}, \dfrac{\partial f(x)}{dx_{2}},...,\dfrac{\partial f(x)}{dx_{n}}),(4)$

$dy = \dfrac{\partial f(x)}{dx}^{T} dx,(5)$

$d(x \pm y) = dx \pm dy, d(xy) = (dx)y + xdy, d(x^{T}) = (dx)^{T},(6)$

$df(x) = f'(x) \odot dx, d(x \odot y) = (dx) \odot y + x \odot dy,(7)$

$d(x^{-1}) = -x^{-1}(dx)x^{-1},(8)$

$xx^{-1} = I， 于是：dxx^{-1} = (dx)x^{-1} + xdx^{-1} = 0，d(x^{-1}) = -x^{-1}(dx)x^{-1}$

$Loss = \dfrac{1}{2} \sum (y_{i}-x_{i}w)^{2} = \dfrac{1}{2} (y-xw)^{2},(9)$

$dLoss = d(y-xw)^{2} = d(y-xw)^{T}(y-xw),(10)$

$d(y-xw)^{T}(y-xw) = d(y-xw)^{T}(y-xw) + (y-xw)^{T}d(y-xw)$

$A = [a_{11}, a_{12}; a_{21}, a_{22};], B = [b_{11}, b_{12}; b_{21}, b_{22};]$

$trace(A^{T}B) = trace( [a_{11},a_{12}; a_{21}, a_{22};]^{T} [b_{11}, b_{12}; b_{21}, b_{22};])$

$trace(A^{T}B) = a_{11}b_{11}+a_{12}b_{12}+a_{21}b_{21}+a_{22}b_{22}$

$dy = \dfrac{\partial f(x)}{dx}^{T} dx = trace(\dfrac{\partial f(x)}{dx}^{T} dx), (12)$

$tr(a) = a, tr(A^{T}) = A, tr(A \pm B) = tr(A) \pm tr(B), tr(AB) = tr(BA)$

$d(y-xw)^{T}(y-xw) = d(y-xw)^{T}(y-xw) + (y-xw)^{T}d(y-xw) \\ = ((y-xw)^{T}d(y-xw))^{T}+ (y-xw)^{T}d(y-xw)$

$tr(((y-xw)^{T}d(y-xw))^{T}+ (y-xw)^{T}d(y-xw)) = tr(((y-xw)^{T}d(y-xw))^{T}) + tr((y-xw)^{T}d(y-xw)) \\ = 2tr((y-xw)^{T}d(y-xw))$

$-(y-xw)^{T}x = -(x^{T}(y-xw)) = x^{T}y-x^{T}xw = 0$

$df = d(a^{T}xb) = a^{T}d(x)b = a^{T}(dx)b$

$tr(a^{T}(dx)b) = tr(ba^{T}dx)$

posted @ 2021-06-22 13:30  ZhiboZhao  阅读(297)  评论(0编辑  收藏  举报