手写数字识别

设神经网络有\(n\)层
第\(i\)层有\(m_i\)个神经元
\(C(...)=\sum_{j=1}^{m_n}(a_j^n-ans_j)^2\)
\(z_j^l=b_j^l+\sum_{k=1}^{m_{l-1}}w_{jk}^la_k^{l-1}\)
\(a_j^l=\sigma(z_j^l)\)

\(\frac{\partial C}{\partial w_{jk}^{l}}\)
\(=\frac{\partial C}{\partial a_j^l}\frac{\partial a_j^l}{\partial z_j^l}\frac{\partial z_j^l}{\partial w_{jk}^{l}}\)
\(=\sigma'(z_j^l)a_k^{l-1}\frac{\partial C}{\partial a_j^l}\)

\(\frac{\partial C}{\partial a_j^l}\)
\(=\sum_{k=1}^{m_{l+1}}\frac{\partial C}{\partial a_k^{l+1}}\frac{\partial a_k^{l+1}}{\partial z_k^{l+1}}\frac{\partial z_k^{l+1}}{\partial a_j^l}\)
\(=\sum_{k=1}^{m_{l+1}}\sigma'(z_k^{l+1})w_{kj}^{l+1}\frac{\partial C}{\partial a_k^{l+1}}\)
\(\frac{\partial C}{\partial a_j^n}=2(a_j^n-ans_j)\)

• 随机设置\(w\)
• 把training随机分成若干个minibatch
• 枚举每个minibatch
  • 枚举minibatch内每个数据，对于每个数据
    • 正着跑一遍把 \(a,z,C\) 算出来
    • 倒着跑一遍把 \(\frac{\partial C}{\partial a},\frac{\partial C}{\partial w}\)算出来
  • 把得到的所有\(\frac{\partial C}{\partial w}\)加起来
  • 调整边权
• 用test集测试