摘要:
h = x @ W1 # x:(N,3) W1:(3,4) h:(N,4) a = tanh(h) # a:(N,4) y = a @ W2 # W2:(4,2) y:(N,2) loss = ... # 假设 dy 已知,形状 (N,2) 试着推出:dW1, da, dh, dW2: dy: sh 阅读全文
摘要:
前向局部导数反向写法 b = a² 2a da = 2*a * db b = 1/a -1/a² da = -1/a**2 * db b = tanh(a) 1 - b² da = (1 - b**2) * db b = exp(a) exp(a) = b da = b * db b = log(a 阅读全文