expRNN

New RNN models:

  1. RNN-orth
  2. EURNN
  3. expRNN
  4. nnRNN

 其中1,2,3都归属于orthogonal RNN

• Long short-term memory (LSTM). (Hochreiter & Schmidhuber, 1997)

• Unitary RNN (URNN). (Arjovsky et al., 2016)

• Efficient Unitary RNN (EURNN). (Jing et al., 2017)

• Cayley Parametrization (SCORNN). (Helfrich et al., 2018)

• Riemannian Gradient Descent (RGD). (Wisdom et al., 2016)

Future work: combine orthogonal RNN with gating. 

posted @ 2020-04-25 14:42  keeps_you_warm  阅读(131)  评论(0)  编辑  收藏  举报